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Executive summary 

There has been a great deal of focus in recent years on high school reform as a way to 
ensure all students graduate ready for a skilled job or additional postsecondary education. 
As expectations for a more highly educated American citizenry rise, what happens in the 
middle grades — the beginning of the secondary-to-postsecondary education continuum — 
matters more now than ever. 

The middle grades are where many students begin to lose ground in key subject areas such 
as mathematics. But the middle grades may also be the last, best chance to identify 
students at risk of academic failure and get them on track in time to succeed in high 
school. Indeed, success in key subjects in the middle grades is a strong predictor of 
success in high school and beyond. 

The past two decades have seen the release of many reports stressing the importance of the 
middle grades. All have focused to one degree or another — with more or less specificity — 
on the broad concepts of developmental responsiveness (to young adolescent needs), 
social equity (ensuring that all students are encouraged and supported to achieve at their 
highest), and academic excellence or rigor (consistent with standards-based instruction) as 
central tenets of their recommendations. 

Only very recently has research and analysis begun to focus on academic outcomes in the 
middle grades, however. To date, very little research has been conducted on the 
relationship between particular middle grade practices and policies and improved 
academic outcomes. 

Although it is widely accepted among educators that most of the difference in student 
outcomes among schools is directly related to student background, it is less widely 
acknowledged that there is great variation in student performance among schools serving 
very similar student populations. This variation is striking — and in many ways, hopeful. It 
makes clear that school and district practices can have a significant impact on student 
outcomes regardless of student background. 

This study contributes to the field by identifying a broad range of traditional and 
newer middle grade policies and practices, and determining in California which of 
these differentiate higher- from lower-performing schools serving similar student 
populations, with performance measured by the state’s standards-based tests. 

Study design 

Based on a sample of 303 middle grade schools located across California, the research 
team conducted three large-scale surveys: of middle grade principals; of English language 
arts and mathematics teachers; and of many of these schools’ district superintendents. In 
total, 303 principals, 3,752 teachers, and 152 district superintendents and five charter 
management organization leaders responded. 

The sample of schools for which the research team analyzed survey data included: 

■ Two distinct subsamples of middle grade schools that served predominantly low- 
income and middle-income students, respectively. 

■ A broad spectrum of school-level academic performance (for the 2008-09 school 
year) within each of the two subsamples of schools. 

■ All major middle grade configurations, including schools serving grades 7-8, 6-8, K- 
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8, and other configurations. 

■ Both charter and traditional public schools. 

Respondents answered survey questions — more than 900 in total among the three 
surveys — developed out of an extensive review of middle grade research, literature on 
principal and district leadership, and related state and federal education policy. Many 
questions were designed to provide a look at the alignment of policy and practice between 
the superintendent, principal, and teacher levels. 

The survey development was guided by ten study domains distilled from the research 
team’s literature and policy review: 

■ Domain A: A positive, safe, engaging school environment. 

■ Domain B: An intense, school-wide focus on improving academic outcomes. 

■ Domain C: School organization of time and instruction. 

■ Domain D: Coherent and aligned standards-based instruction and curricula. 

■ Domain E: Extensive use of data to improve instruction and student learning. 

■ Domain F: Early and proactive academic interventions. 

■ Domain G: Attention to student transitions. 

■ Domain H: Teacher competencies, evaluation, and support. 

■ Domain I: Principal leadership and competencies. 

■ Domain J: Superintendent leadership and district support. 

For the purposes of analysis, the research team grouped the superintendent, principal, and 
teacher survey items into subdomains of practice within each of the ten study domains. 
These practice subdomains served as the independent variables for analysis. 

School-level outcomes from seven California Standards Tests (CSTs) served as the 
study’s dependent variables: 

■ The Grade 6 English Language Arts CST. 

■ The Grade 7 English Language Arts CST. 

■ The Grade 8 English Language Arts CST. 

■ The Grade 6 Mathematics CST. 

■ The Grade 7 Mathematics CST. 

■ 8th graders’ outcomes on the General Mathematics CST. 

■ 8th graders’ outcomes on the Algebra I CST. 

The research team analyzed the relationship between reported practices and school-level 
outcomes on each of these seven CSTs in two different ways: 

■ Cross-sectional analyses measured the power of each of the ten research domains, and 
the subdomains within them, in explaining variation in school-level mean CST 
outcomes in 2009, controlling for student demographics and other school variables. 

■ Longitudinal analyses measured the power of each of the domains, and the 
subdomains within them, in explaining variation in school-level mean student growth, 
by controlling for prior student achievement. These analyses used a data file with test 
scores over four years for the students in the sample to generate a 2009 “predicted” 
mean test score for the students in each school, based on their prior achievement. These 
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analyses identified which school and district practices were most strongly associated 
with 2009 school-level mean CST scores above the predicted levels. 

These cross-sectional and longitudinal regression analyses revealed: 

■ Which domains of practice and policy most strongly predict higher school-level CST 
outcomes; and 

■ Which of the many subdomains within each domain correlate positively with these 
outcomes at the .05 level of statistical significance. 

Findings 

The ten research domains fell into three “groupings” with respect to their relative 
predictive strength in differentiating higher-performing middle grade schools in the 
sample: 

■ The domain with the greatest predictive strength across most of the cross-sectional and 
longitudinal analyses was Domain B, “An intense, school-wide focus on improving 
academic outcomes.” 

■ Six domains consistently followed Domain B in terms of predictive strength. Relative 
to one another, their predictive strength varied depending on the analysis, and there 
were relatively few statistically significant differences between them. These domains 
pertain to standards-based instruction and curricula; extensive use of data; proactive 
academic interventions; teacher competencies; principal leadership; and superintendent 
leadership/district support. 

■ Three domains, although they did differentiate schools with higher CST scores and 
higher gains in CST scores, did so with less predictive strength than the other seven 
domains in almost every analysis conducted. These three domains pertain to school 
environment; the organization of time and instruction; and attention to student 
transitions. 

After accounting for specific school policies and practices, no single grade configuration 
was consistently associated with higher performance on California’s standards-based 
tests in English language arts and mathematics in this study. Both more effective and less 
effective policies and practices were found in schools with every grade configuration 
studied. 

Although identifying the relative predictive strength of these three domain groupings is an 
important finding, equally important — if not more so — are the specific district- and 
school-level practices under each domain that distinguished higher-performing middle 
grade schools. 

The power of this study is in the specifics. Readers hoping to get the most from this 
Initial Research Report are encouraged to spend time considering the specific policies and 
practices highlighted among the study’s findings as differentiating higher- from lower- 
performing middle grade schools in the sample. 

For example, Domain B — “An intense, school-wide focus on improving academic 
outcomes” — includes a substantial number of reported policies and practices, such as: 

■ The school’s curriculum and instruction are designed to prepare students for a 
rigorous high school curriculum. 

■ The school sets measurable goals for improving district benchmark test scores. 

■ The principal regularly communicates to faculty the importance of high expectations 
for the achievement of all students. 
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■ Instructional time in the classroom is protected from unnecessary interruption. 

■ The school regularly communicates to students the importance of middle grade 
achievement to their future and of taking responsibility for their learning, and 
communicates to parents the importance of their role in setting high expectations for 
students’ academic success. 

■ The school has requirements or contracts for parent participation. 

Readers will find these specifics presented in two places. The policies and practices 
are detailed, by domain, in the Findings section of this report. In addition, the 
specific survey items on which these findings are based are reproduced in Technical 
Appendix C, available from http://www.edsource.org/middle-grades-study.html. 

Taken together, the subdomains that differentiated higher-performing schools in the 
various domains tell a coherent and compelling story. Other things being equal, school- 
level achievement in the 303-school sample tended to be higher to the extent that 
superintendents, principals, and middle grade English language arts and mathematics 
teachers strongly agreed on a clear, consistent, and intense focus on improving student 
academic outcomes. 

Further, the combined findings suggest a range of interrelated practices through which this 
focus happens. For example: 

■ There is a strong organizational press toward improving student outcomes, with 
superintendents, principals, and teachers emphasizing and being evaluated based in 
part on such improvements. 

■ Middle grade educators closely align curricula, instruction, and assessments with the 
state’s K-12 academic content standards in core subjects. From the district to the 
principal and the teachers, student assessment and other data are used extensively to 
evaluate and improve teacher practice and student outcomes. 

■ Middle grade educators have a future orientation toward preparing students to succeed 
in a rigorous high school curriculum. In other words, educators’ focus on student 
outcomes is not geared only toward improving student achievement as measured by 
standards-based exams. 

■ Perhaps in recognition that they have only a few years to work with their students, 
middle grade educators implement comprehensive and targeted academic programs to 
identify quickly and intervene proactively with students who are two or more years 
behind grade level, are English learners, or are at risk of failure in the current school 
year. 

■ A strong and cohesive professional culture marshals and focuses time and other 
resources intently on improving middle grade student outcomes. In particular: 

• The superintendent establishes an academic vision and follows through with 
accountability, while at the helm of an effectively functioning district that provides 
middle grade schools with comprehensive support aimed at improved student 
outcomes. 

• Principals provide strong leadership to drive and orchestrate a schoolwide focus on 
improved student outcomes. 

• Teachers work collectively as a team and individually in their classrooms on a 
shared mission to improve student outcomes and prepare students for success in 
high school. 

Notably, such practices and policies generally differentiated higher- from lower- 
performing middle grade schools in the sample regardless of whether schools served 



4 



Initial Research Report — Gaining Ground in the Middle Grades 



predominantly low- or middle-income students. Equally important, these practices can be 
adopted and implemented by any district, principal, and school community of teachers. 
They are not dependent upon grade configuration or internal classroom organization. 

Certainly, the California Standards Tests in English language arts and mathematics do not 
measure many other important things that middle grade students should be learning at 
school — such as art, social studies, science, and music, as well as citizenship and tolerance 
of differences. 

But scores on these tests do provide middle grade students and teachers, their school 
districts, and the state with a consistent way to measure the progress students are making 
toward mastering the important mathematics, reading, and writing skills that will enable 
them to succeed academically in high school and beyond. 

Across California — and no doubt nationwide — schools serving similar student populations 
can vary widely in how well their students perform on such tests. This study shows that, 
although the socioeconomic backgrounds of students are one strong predictor of school- 
level academic achievement, the practices and policies enacted by middle grade educators 
also have a significant relationship with these outcomes. Their focus on their middle grade 
mission, and the resources they have available to pursue their goals, can make a 
difference. The interrelated practices identified in this study may help middle grade 
schools and districts — in California and nationally — continue their efforts to improve 
students’ academic outcomes at a critical time in their academic careers. 
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I. Introduction: Why the middle grades? 

There has been a great deal of focus in recent year's on high school reform as a way to 
ensure all students graduate ready for a skilled job or additional postsecondary education. 
As expectations for a more highly educated American citizenry rise, what happens in the 
middle grades — the beginning of the secondary-to-postsecondary education continuum — 
matters more now than ever. 

■ The middle grades are where many students begin to lose ground in key subject areas 
such as mathematics. In California — which educates one in every eight students in 
grades 6 through 8 in the United States — student achievement in mathematics in any 
given year is lower in the middle grades than in the elementary grades, as measured by 
the state’s standards-based tests. Although California has made much progress in 
recent years in improving the performance of its middle grade students in the subject, 
two-thirds of 4th graders in the state scored proficient or advanced in mathematics in 
2009, compared with only 43% of 7th graders, according to data published by the 
California Department of Education (CDE). Nationally, most states see a dip in middle 
grade proficiency (i.e., the percentage of students scoring “at” or “above proficient”) 
compared with elementary levels, as measured by the National Assessment of 
Educational Progress (NAEP). 

■ The middle grades are the last, best chance to identify students at risk of academic 
failure and get them on track in time to succeed in high school. Recent studies in 
California (Kurlaender, Reardon, et ah, 2008; Zau and Betts, 2008) and elsewhere 
(Balfanz, Herzog, et ah, 2007; Balfanz, 2009) show that many students at the greatest 
risk of high school failure could be identified early — during the middle and even late 
elementary grades — by their grades, attendance, behavior, and test scores. For 
example, Balfanz and colleagues found that the middle grade experience of students in 
high-poverty Philadelphia schools strongly influences their odds of graduating from 
high school. The more “off track” indicators a student has and the fewer opportunities 
the school provides for intervention and correction, the more likely that student will not 
graduate from high school, much less enter a college preparatory curriculum there. 

■ Success in key subjects in the middle grades is a strong predictor of success in high 
school and beyond. In California, students’ scores on middle grade standards-based 
tests in English language arts and mathematics can predict whether they will pass the 
California High School Exit Exam, which students first take in grade 10. Further, 
students’ grades and test scores in mathematics help determine their course placement 
in the subject when they begin high school. This has implications for science 
placements as well, and consequently for students’ prospects of completing the college 
preparatory coursework needed to be eligible for admission to the University of 
California and the California State University systems. 

The past two decades have seen the release of many reports stressing the importance of the 
middle grades. All have focused to one degree or another — with more or less specificity — 
on the broad concepts of developmental responsiveness (to young adolescent needs), 
social equity (ensuring that all students are encouraged and supported to achieve at their 
highest), and academic excellence or rigor (consistent with standards-based instruction) as 
central tenets of their recommendations. 

But only very recently has research and analysis begun to focus on academic outcomes in 
the middle grades. To date, very little research has been conducted on the relationship 
between particular' middle grade practices and policies and improved academic outcomes. 
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BACKGROUND 

The California context 

California’s 1.5 million 6th-, 7th-, and 8th-grade public school students represent one out of 
eight middle grade students in the United States. Based on these numbers alone, California is 
important; but its student diversity and variety of school and district configurations add to the 
state’s relevance for any consideration of middle grade education. 

Grade and district configurations in California vary widely: 

■ Among more than 2,000 schools in California with grade spans that included 7th and 8th 
grade in 2007-08 (CDE, 2008b, pg. 3), about: 

• 35% were K-8 schools; 

• 41% were 6-8 schools; 

• 1 7% were 7-8 schools; and 

• 7% were a variety of other configurations. 

■ In 2008-09, California was home to 1 ,043 school districts and 746 charter schools. 

■ California’s middle grade schools are distributed across three different kinds of school 
districts: K-8 elementary districts, K-12 unified districts, and high school districts (most of 
which serve grades 9-12). Although the vast majority of middle grade students in the state 
are enrolled in elementary or unified districts, a small number of high school districts also 
educate 7th and 8th graders. 

■ The school districts serving California’s middle grade students range from very small 
districts with one school and one person who serves as both school principal and district 
superintendent — i.e., superintendent-principals — to Los Angeles Unified School District 
(LAUSD), which served more than 687,500 K-12 students in 2008-09. 

Middle grade education in California is also influenced by the incredible diversity of the state’s 
students: 

■ More than 40% of middle grade students in the state live in poverty (CDE, 2008b, pg. 3). 

■ Statewide across all grades in 2008-09, 49% of California students were Latino, nearly 
28% were white, 8.4% were Asian, 7.3% were African American, and the balance were of 
various other ethnic backgrounds. The ethnic distribution of students throughout the state 
is uneven. For example, more than 40% of African American students in California attend 
public schools in just 1 3 school districts. 

■ One in five middle grade students in the state is categorized as an English learner (CDE, 
2008b, pg. 3). Across all grades, about 85% of English learners (ELs) speak Spanish, and 
the rest speak a plethora of other languages. California’s EL students are distributed 
unevenly throughout the urban, rural, and suburban areas of the state. EL students are 
over-represented in Los Angeles County, for example, but are also more than 30% of 
students in a few small, largely rural counties in the agricultural area of the state. 

Although California serves the highest proportion of English learners in the nation and a 
disproportionate number of low-income students compared to the national average, the 
resources available to California schools are fewer than those in most other states. In addition, 
to the extent that California has invested in education reform in the past 15 years or so, that 
investment has largely centered on the early elementary grades and high school. 

For 30 years, California has lagged behind the national average in its expenditures per pupil. 

In 2006-07, California ranked 49th among all states (and Washington D.C.) in its ratio of 
teachers to students, at about 74% of the U.S. ratio. Further, California’s ratio of school site 
administrators to students was 71 % of the U.S. ratio, and that of district officials was 39% 
(EdSource, 2009b). At the same time — given the state’s relatively high cost of living — average 
educator salaries are the highest in the country. 

(Data in this box are from the California Department of Education and the National Center for Education 
Statistics.) 
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Such questions are vital. The past few years have witnessed a global and national 
economic crisis. At the same time that public expectations for a better-educated workforce 
are rising, state budgets to provide funding for K-12 education have been cut across the 
nation, and in few places more than in California. While education funding from states is 
declining, federal education funding opportunities are increasing and civic groups are re- 
examining options to raise funds through other sources. We appear to be entering a “new 
normal,” where for many years ahead policymakers and educators will have to do more 
with less. On the upside, the crisis is causing individuals and institutions alike to rethink 
and justify their priorities and to be open to new goals and new ways of accomplishing 
important objectives. 

Middle grade educators and their students, and their efforts and success, are important. 
This study contributes to the field by identifying a broad range of traditional and newer 
middle grade policies and practices, and determining in California which of these 
differentiate higher- from lower-performing schools serving similar student populations, 
with performance measured by the state’s standards-based tests. 
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II. The research question: 



What school and district practices and policies 
differentiate higher- from lower-performing schools 
serving demographically similar middle grade students? 

Although it is widely accepted among educators that most of the difference in student 
outcomes among schools is directly related to student background, it is less widely 
acknowledged that there is great variation in student performance among schools serving 
very similar student populations. This variation is striking — and in many ways, hopeful. It 
makes clear that school and district practices can have a significant impact on student 
outcomes regardless of student background. 

Figure 1 illustrates this clearly. This scatterplot shows the mean scale scores of California 
middle grade schools on the Grade 8 California Standards Test (CST) in English language 
arts in 2008, plotted against each school’s score on the statewide Schools Characteristics 
Index (SCI), which serves as a proxy for student socioeconomic status. (For more 
information, see the California’s School Characteristics Index box on page 23.) 



Figure 1: Average Grade 8 English Language Arts CST scale scores among California 
schools serving at least grades 7 and 8 
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Available at: http://star.cde.ca.gov/star2008/ EdSource, 2/10 
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Figure 1 illustrates both of the patterns just highlighted. First, as expected, a higher SCI 
correlates with higher schoolwide student achievement on the California Standards Tests 
(CSTs). Student test scores increase as family education and socioeconomic status 
increase. 

But Figure 1 also shows that average student outcomes on the CSTs vary widely — even 
among middle grade schools serving student populations with very similar backgrounds 
and demographic profiles. As the figure shows: 

■ Among middle grade schools located in the 20th-35th percentile range (or “band”) of 
the SCI, which serve students from predominantly lower-income families, school-level 
mean scale scores on the 2008 Grade 8 English Language Arts CST varied by 79 
points, ranging from 298 to 377 (on a scale from 150 to 600). 

■ Among middle grade schools located in the 70th-85th percentile band of the SCI, 
which serve students from predominantly middle-income families, school-level mean 
scale scores on the test varied by 92 points, ranging from 307 to 399. 

In comparison with these broad variations, the average scores on the test for each of these 
two SCI bands overall differed by only about 32 points (328.9 vs. 360.6 points in 2008). In 
other words, there is more variation in school-level student achievement on the Grade 8 
English Language Arts CST within each of the two SCI bands than, on average, exists 
between them. This pattern recurs with respect to the other English language arts and 
mathematics CSTs taken by middle grade students. 

This study is designed, first and foremost, to identify actionable school and district 
practices and policies that differentiate higher- from lower-performing middle grade 
schools (as measured by California’s standards-based tests) serving similar students. 

Secondarily, the study is also designed to shed light on the middle grade policies and 
practices currently in use in California schools, and on whether practices that differentiate 
higher-performing schools vary based on whether schools serve predominantly low- or 
middle-income students. 
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III. Study design: A large-scale survey of 
middle grade schools 

Summary of the study's architecture 

The study design required to answer these research questions was complex. Figure 2 
below shows the overall organization, complexity, and scale of the study design. 



Figure 2: Overall design of the study 




The research team conducted a large-scale survey of middle grade principals, and English 
language arts and mathematics teachers, at schools across California — as well as a survey 
of many of these schools’ district superintendents. 

The sample of schools for which the research team analyzed survey data: 

■ Was bimodal , including middle grade schools from: 

• The 20th-35th percentile SCI band, serving predominantly low-income students, 
and 

• The 70th-85th percentile SCI band, serving predominantly middle-income 
students. 
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■ Included a/a// spectrum of school-level academic performance for the 2008-09 
school year within each of those two SCI bands. 

■ Included all major middle grade configurations , including schools serving grades 7-8, 
5/6-8, K-8, and other configurations. 

■ Included both charter and traditional public schools. 

Respondents for these schools provided answers to survey questions — more than 900 in 
total — developed out of an extensive review of middle grade research and policy 
literature, as well as literature on principal and district leadership, tailored to California’s 
education policy context. The survey development was guided by ten study domains 
distilled from this literature review. The survey focused on neutrally -phrased, concrete, 
actionable practices and policies. In order to enable a view into the alignment of policy 
and practice between the superintendent, principal, and teacher levels, a portion of the 
questions were asked of at least two, if not all three, of these actors. 

For the purposes of analysis, the research team grouped the superintendent, principal, and 
teacher survey items into subdomains of practice within each of the ten study domains. 
These practice subdomains served as the independent variables for analysis. 

School-level outcomes from seven California Standards Tests (CSTs) served as the 
study’s dependent variables: 

■ The Grade 6 English Language Arts CST. 

■ The Grade 7 English Language Arts CST. 

■ The Grade 8 English Language Arts CST. 

■ The Grade 6 Mathematics CST. 

■ The Grade 7 Mathematics CST. 

■ 8th graders’ outcomes on the General Mathematics CST. 

■ 8th graders’ outcomes on the Algebra I CST. 

The research team analyzed the relationship between reported practices and school-level 
outcomes on each of these seven CSTs in two different ways: 

■ Cross-sectional analyses measured the power of each of the ten research domains, and 
the subdomains within them, in explaining variation in school-level mean CST 
outcomes in 2009, controlling for student demographics and other school variables. 

■ Longitudinal analyses measured the power of each of the domains, and the 
subdomains within them, in explaining variation in school-level mean student growth, 
by controlling for prior student achievement. These analyses used a data file with test 
scores over four years for the students in the sample to generate a 2009 “predicted” 
mean test score for the students in each school, based on their prior achievement. These 
analyses identified which school and district practices were most strongly associated 
with 2009 school-level mean CST scores above the predicted levels. 

The rest of this section describes the study design in more detail, with particular attention 
to: 



■ The research domains and survey instruments at the core of the study; 

■ How the research team defined the population of middle grade schools eligible to 
participate in the survey, the process by which the team recruited school principals and 
teachers — as well as districts leaders — to participate, and the resulting sample; and 
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■ How survey responses for these schools were analyzed against school-level CST 
outcomes, through both cross-sectional and longitudinal analyses, to determine which 
areas of practice and policy differentiate higher-performing middle grade schools. 

The research and review process 

Ten broad conceptual domains guide the research team’s survey of middle grade schooling 
practices and policies in California. The domains and surveys were developed based on 
extensive review of the available research and policy literature. This included the myriad 
reports issued in the past several decades on middle grade reform and more limited 
outcomes-based research at the middle grade level. The research team also considered 
current education policy in California and followed state and national dialogue about such 
issues as algebra and literacy in the middle grades. 

The research team’s previous study of effective school policies and practices in California 
elementary schools (Williams, Kirst, Haertel, et al., 2005) also informed the domains and 
surveys. That study drew from research related to “effective schools” practices in the 
standards -based era. The research team updated its understanding of these issues through 
attention to more recent literature on such topics as the use of technology to support data- 
informed decision-making, the early identification of students in need of assistance, and 
the role of district and school leadership. 

In addition, a range of national and state experts on middle grade education reviewed and 
provided feedback to the research team on the domains and survey instruments. Three 
national experts were consultants to the study and the survey development process: 

■ Robert Balfanz, principal research scientist at the Everyone Graduates Center, Johns 
Hopkins University. 

■ Hayes Mizell, distinguished senior fellow of the National Staff Development Council 
and former director of the Edna McConnell Clark Foundation’s program for student 
achievement. 

■ Uri Treisman, professor of mathematics and director of the Charles A. Dana Center, 
University of Texas at Austin. 

Other experts also provided the research team with review and feedback during the 
development process, including individuals from: 

■ The California Department of Education (CDE) Middle and High School Improvement 
Office. 

■ The CDE Curriculum Frameworks and Instructional Resources Division. 

■ The California Commission on Teacher Credentialing (CTC). 

■ The California Subject Matter Project (CSMP). 

■ NewSchools Venture Fund. 

■ Leaders and teachers from Aspire Public Schools. 

■ Current California superintendents and middle grade principals. 

■ Current middle grade teachers who provided feedback on the teacher survey instrument 
during pilot sessions at two middle schools, located respectively in northern and 
southern California. 

The research team solicited additional feedback on aspects of the domains and survey 
instruments pertaining to the middle grade education of English learner (EL) students and 
English language development (ELD) practices. Robert T. Linquanti, project director for 
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English Learner Evaluation and Accountability Support (ELEAS) at WestEd, worked on 
contract with EdSource staff on the development of a framework related to instructional 
strategies and local policies for the education of EL students in the middle grades. A group 
of EL practitioners and researchers then prioritized the items included in the framework. 
Some of these experts participated in a conference call to discuss the framework, with a 
focus on policies and practices they believed would vary among schools and might 
correlate with middle grade CST outcomes in English language arts and mathematics. The 
research team also conferred with Aida Walqui, director of teacher professional 
development at WestEd, at several points during the study for feedback on the survey 
instruments and important context for interpreting findings. 

The ten research domains 

This section introduces the ten conceptual domains that guided the development of the 
survey instruments and the analyses. Each introduction discusses briefly: 

■ The relevance of the domain for middle grade education in California, as informed by 
the available research and policy literature; and 

■ Key areas of practice and policy encompassed by the domain, into which the survey 
instruments inquired. 

The reader will note that certain topics cut across domains. In particular, elements of 
principal and district leadership are considered as they pertain to both a particular domain 
of practice and policy (e.g., as a component of “An intense, school-wide focus on 
improving academic outcomes”) and the domains focused on principal or district 
leadership. 

Practices and policies pertaining to the support of English learners — and to ensuring these 
students’ access to core middle grade content in English language arts and mathematics — 
are considered across the domains. The aspects of the domains focused on EL students 
were informed greatly by additional expert review (described earlier) and relevant 
research and policy literature (e.g., Meltzer and Hamann, 2004, 2005; Short and 
Fitzsimmons, 2007; Abbott and Ganahl, 2008; WestEd, 2008). 

Domain A-A positive, safe, engaging school environment 

Responsiveness to the developmental needs of young adolescents, strong adult-student 
relationships, and student engagement are signature issues of the “middle school 
movement” that drove middle grade reform for several decades. 

This emphasis has been driven, in part, by worry about student disengagement and 
countervailing influences that pull students away from school (e.g., Carnegie Council on 
Adolescent Development, 1989; see also Juvonen, Le, et al., 2004). Although some have 
criticized over-emphasis on developmental responsiveness at the expense of academic 
achievement (e.g., Bottoms, Cooney, et al., 2003; Yecke, 2005), the concern for engaging 
young adolescents in relevant activities is a fundamental aspect of most middle grade 
reform recommendations (e.g., CDE, 2008d; National Forum to Accelerate Middle-Grades 
Reform and California Middle Grades Alliance, 2009; National League of Middle 
Schools, 2010), and is typically viewed as one necessary condition for student success that 
complements a school’s academic focus (e.g., see Lee, Smith, et al., 1999). 

Student behavior and attendance — both important indicators of positive engagement — are 
particularly important in light of research showing that behavior and attendance can 
predict, for example, whether students entering high-poverty middle schools behind grade 
level in mathematics will catch up before they leave. This research emphasizes the value 
of an immediate response to a student’s first unexcused absence and consistent modeling 
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and recognition of positive student behavior (e.g., Balfanz and Byrnes, 2006; Balfanz, 
Herzog, et al., 2007). 

This study’s survey instruments inquired into such strands of policy and practice as: 

■ Reinforcing positive middle grade student behavior. 

■ Ensuring a safe campus environment. 

■ Developing strong staff-student relationships. 

■ Ensuring student attendance. 

■ Providing extracurricular activities and elective courses. 

■ Utilizing engaging instructional practices. 

Domain B-An intense, school-wide focus on improving academic outcomes 

Standards-based accountability makes raising student achievement in the middle grades 
and preparation for high school, as measured publicly against high academic standards and 
expectations, non-negotiable priorities. Criticism of the middle grades is often driven by 
concern that schools devote inadequate attention to building a pervasive culture of 
academic achievement (e.g.. Bottoms, Cooney, et al., 2003; Yecke, 2005). In addition, 
some recent research suggests that many California students, by the end of grade 9, are 
already off-track for meeting the minimum course-taking requirements necessary to be 
eligible for admission to a public four-year state university (Finkelstein & Fong, 2008). 

All involved in the education of middle grade students have a role to play in cultivating 
and sustaining a pervasive focus on academic achievement. For example: 

■ Districts can prioritize and set explicit goals for student achievement (e.g.. Bottoms, 
Cooney, et al., 2007). 

■ Principals can cultivate collective responsibility for school improvement and a shared 
vision for academic achievement (e.g., Cotton, 2003; Feithwood, Steinbach, et al., 
2002; Feithwood and Riehl, 2003; National Association of Secondary School 
Principals, 2006). 

■ Parents can play a key role in socializing students into future goals and aspirations that 
place a high value on educational achievement (e.g., Hill and Tyson, 2009; Juvonen, 

Fe, et al., 2004). 

■ Students can also take responsibility for academic achievement. For example, one 
recent study found that academic behaviors such as “academic discipline” — student 
planning and organization, follow-through, and sustained effort — are important for 
course success in grade 8 and grade point average in grade 9, and can provide 
educators with important indications of which students need intervention (ACT, 2008). 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ Setting measurable goals for student achievement — for example, with respect to federal 
and state accountability measures, CSTs, and district benchmarks. 

■ Setting priorities and measurable goals related to Algebra I course-taking and success. 

■ Focusing on preparing students for a rigorous high school curriculum. 

■ Focal educators prioritizing and holding themselves accountable for student 
achievement. 

■ Communicating to students and parents the importance of taking responsibility for 
academic achievement. 
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Domain C-School organization oftime and instruction 

Time — for additional instructional support, for common planning and collaboration, and 
so forth — is an important resource for middle grade educators. In addition, how middle 
grade school leaders configure the master schedule and organize relationships among staff 
and students provide important contexts for how the school meets its academic and 
developmental goals (e.g., National Association of Secondary School Principals, 2006; 
Short and Fitzsimmons, 2007). 

For example, middle grade research and reform literature frequently cites structural 
practices such as flexible scheduling, vertical looping (i.e., students have the same 
teachers for two or more years), interdisciplinary team teaching, and small learning 
communities as ways to enrich the interactions of students with adults, and educators with 
one another. Such practices are cited variously in middle grade policy reports and 
statements of educational philosophy (e.g., Carnegie Council on Adolescent Development, 
1989; Juvonen, Le, et al., 2004; National Forum to Accelerate Middle Grades Reform, 
2004; National Association of Secondary School Principals, 2006; National Middle 
School Association, 2010), as well as middle grade research (Flowers, Mertens, and 
Mulhall, 1999, 2003) and various comprehensive school reform models (e.g., Felner, 
Jackson, et al., 1997; Balfanz, Ruby, et al., 2004; Maclver, Ruby, et al., 2007). 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ How much time the school provides for instruction in mathematics and English 
language arts, and whether this varies based on a student’s academic proficiency. 

■ Extra instructional time for students with less-than-reasonable fluency in English. 

■ How periods or blocks of time are configured in the master schedule and whether it is 
flexible. 

■ Classroom organization (i.e., departmentalized, self-contained, or semi-self-contained). 

■ Special strategies for grouping students and teachers such as team teaching, vertical 
looping, and small learning communities. 

■ Common planning time for teachers to collaborate within subject departments and/or 
grade levels to discuss student needs and improve practice. 

Domain D-Coherentand aligned standards-based instruction and curricula 

Standards-based education is premised on the idea that high academic standards for all 
students should guide coherent educational programs. Current efforts to articulate 
common core standards nationwide, and regional efforts to encourage greater rigor in 
middle grade standards, assessments, and performance benchmarks in mathematics and 
reading (e.g., Southern Regional Education Board, 2009c), show this remains an ongoing 
concern. 

The alignment of educational practice with high academic standards has been a core 
aspect of California’s approach to middle grade education since the adoption of the state’s 
academic content standards. Taking Center Stage (CDE, 2001b) — now in its second 
incarnation as an online resource for middle grade educators (CDE, 2008d) — called on 
middle grade educators to fully embrace the state’s academic content standards as a 
common basis for the quality education of all middle grade students. These standards form 
the basis for instructional materials adopted by the state for grades K-8. 

Adolescent literacy is a curricular topic of particular current interest (e.g., Kamil, 2003; 
Biancarosa & Snow, 2006; Heller and Greenleaf, 2007; WestEd, 2008; Southern Regional 
Education Board, 2009a; Carnegie Council on Advancing Adolescent Literacy, 2010). 
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BACKGROUND 



Standards-based education in California's middle grades 

California was an early adopter of standards-based education reform. By 1999, California had 
adopted statewide academic content standards in English language arts, mathematics, 
history/social science, and science. The standards in English language arts and 
mathematics — the two subject areas considered in this study — have been rated highly by 
some national organizations (e.g., Klein, Braams, et al., 2005; Stotsky, 2005; American 
Federation of Teachers, 2008), but have also stirred controversy. California’s adoption of its 
K-12 academic content standards in mathematics, for example, revealed philosophical 
tensions about the extent to which the standards should emphasize basic operational skills 
and mathematical precision or conceptual understanding and practical relevance. 

Although technically voluntary, the standards in English language arts and mathematics have 
been infused through most aspects of California’s public education system, creating a strong 
incentive for schools to adopt them. For example, the standards form the basis for annual 
standards-based testing and for the state’s periodic adoption of curriculum programs for 
grades K-8, from which local districts choose when using state-provided funds for instructional 
materials. 

In English language arts, these include grade-level content standards for grades 6-8. In 
mathematics, grade-level content standards are established for grades 6 and 7. Beginning in 
grade 8, however, the mathematics standards are organized by discipline, beginning with 
Algebra I. 

Statewide standards-based testing — the centerpiece of which is the criterion-referenced 
California Standards Tests (CSTs) — follows a similar pattern in the middle grades. Seven of 
these tests provide the middle grade outcome variables for this study, discussed in more detail 
later. 

■ In English language arts, California administers a grade-level CST in each of grades 6-8, 
which nearly all students at each grade level take. 

■ In mathematics, the state administers a grade-level CST in each of grades 6 and 7. 

■ Beginning in grade 8 — or a year earlier in the case of more advanced 7th graders who take 
Algebra I — students take different mathematics CSTs depending on the courses in which 
they enroll. California 8th graders who are enrolled in a full Algebra I course — 54% in 
2009 — take the Algebra I CST. A few more advanced 8th graders take more advanced 
tests such as the Geometry CST. Middle grade schools in California vary widely in the 
proportion of 8th graders who take these tests, however. In 2009, 39% of California 8th 
graders took the General Mathematics CST instead, which assesses these students’ 
achievement in meeting primarily grade 7 standards. These 8th graders were enrolled in a 
course below Algebra I, or in the first year of a two-year Algebra I course. 

California’s practice of end-of-course testing in mathematics in grade 8 pre-dates and differs 
from the federal No Child Left Behind law, which assumes that math testing in grade 8 will be 
based on a state’s content standards for that grade. Recently, this difference between state 
and federal policy set the stage for a controversial July 2008 motion by the California State 
Board of Education — which was subsequently blocked — that called for the state’s Algebra I 
CST to become the sole mathematics test of record for federal accountability purposes in 
grade 8 (for more information, see EdSource, 2009a). 

Unlike such tests as the National Assessment of Educational Progress (NAEP), which include 
both multiple-choice and constructed-response questions, the CSTs taken by middle grade 
students in English language arts and mathematics are entirely multiple choice, with the 
exception of a writing prompt included for grade 7 English language arts. 
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This topic is important because middle grade students do not read and write in school 
primarily as an end or basic skill in itself, but to learn new academic content, including in 
disciplines with varying conventions for organizing information and making arguments. 

In addition, students enter middle grade schools with sometimes widely varying levels of 
academic preparation. Schools must consider carefully how to group and place students 
most effectively in their classes. This issue has received particular attention recently — and 
has been a topic of controversy in California — as it pertains to student placement in 
Algebra I in grade 8 (e.g., see Burris, Heubert, et ah, 2006; Loveless, 2008; Kriegler and 
Lee, 2008; EdSource, 2009a). 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ Adoption and ongoing review of schoolwide instructional improvement plans. 

■ Decisions about English language arts and mathematics curricula, and expectations for 
use. 

■ Factors that schools consider when placing students into English language arts and 
math classes, including Algebra I. 

■ The extent to which teachers coordinate curricular scope, sequence, and pacing. 

■ Literacy across the curriculum. 

Domain E-Extensive use of data to improve instruction and student learning 

Use of assessment and other data to reflect on and hold schools accountable for student 
progress is a core feature of standards-based education. This means schools must be 
effective learning organizations, capable of reflecting continuously on what is working, 
what is not, and how educators might change course — and that requires effective use of 
data to inform decision-making. In addition, some recent research shows that the middle 
and even late-elementary grades are a crucial time in which educators might identify 
students who are at particular risk of later academic failure and even dropping out of 
school (e.g., Jerald, 2006; Balfanz, Herzog, et ah, 2007; Kurlaender, Reardon, at ah, 2008; 
Rumberger and Lim, 2008; Zau and Betts, 2008). This research draws attention not only to 
test scores and grades, but also to behavior and attendance data. 

Districts play a crucial role in making ongoing reflection on progress possible at the 
school level (Massell, 2000; Fairman and Firestone, 2001; Knapp, Copland, and Talbert, 
2003; McLaughlin and Talbert, 2003; Petrides, Nodine, et ah, 2005; Bottoms, Cooney, et 
al., 2007; Augustine, Gonzalez, et ah, 2009). Districts do this by enabling schools to 
access student assessment data quickly and in a usable form, and by providing for 
reflection on common benchmarks for student learning. 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ Review of behavior and attendance data. 

■ Principals’ use of data to inform instructional improvement and their role in ensuring 
that teachers use data in common planning. 

■ Teachers’ access to and use of student achievement and assessment data, including for 
English learner (EL) students. 

■ Teachers’ use of different kinds of assessments in the classroom and their use of 
assessment data to inform instruction. 

■ The district’s support for the school’s capacity to use data, including benchmark data, 
and its support for diagnostic assessments. 
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Domain F-Early and proactive academic interventions 

Reflecting on school progress and identifying middle grade students who need additional 
support is only the first step in keeping these students engaged in school. Schools must 
also have practices and policies in place to act quickly on this information. This requires 
practices for ongoing instructional support and/or interventions for students who are 
behind grade level. This also raises questions about the effective coordination of subject- 
area instruction with other instructional services, such as English language development 
(e.g., Short and Fitzsimmons, 2007). 

Recent research and policy literature emphasize early action to ensure that students, 
including English learners, quickly receive any additional support they need in flexible 
ways that do not disqualify them from later academic opportunities (e.g., Balfanz, 
McPartland, et ah, 2002; Bottoms, Cooney, et ah, 2003; Balfanz, Ruby, et ah, 2004; 
Southern Regional Education Board, 2005; Short and Fitzsimmons, 2007; CDE, 2008d). 
For example, schools applying for recognition as middle grade “Schools to Watch” in 
California are expected to assess the extent to which “accelerated, short-term interventions 
for students with similar needs are fluid and do not become low-level or permanent tracks” 
(National Forum to Accelerate Middle-Grades Reform and California Middle Grades 
Alliance, 2009, pg. 7). 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ District and principal assignment of teachers to ensure that students with the most need 
are served well. 

■ Teacher collaboration to coordinate interventions and English language development 
with regular instruction. 

■ A variety of intervention strategies for students identified for additional support — 
required and voluntary. 

■ The placement of non-newcomer EEs and primary language support for ELs in the 
classroom. 

Domain G — Attention to student transitions 

Depending on a school’s grade configuration, middle grade educators may be responsible 
for managing two key institutional transitions in students’ lives: from elementary school to 
the middle grades, and from the middle grades to high school. 

Discussion of student transitions is closely linked to debates about school grade 
configuration. Some argue that the institutional transition from elementary to middle 
school is disruptive to early adolescent development (e.g., Juvonen, Le, et ah, 2004; 

Eccles, 2008) — a concern also raised about the transition into high school (e.g., Alspaugh, 
1998a). In light of these concerns, the California Department of Education’s Taking 
Center Stage, Act II (2008d) cites seamless student transitions, with clear academic and 
behavioral expectations, as a key form of developmental responsiveness. 

Data are important for facilitating the transition to middle school. The National 
Association of Secondary School Principals encourages middle grade staff to “learn 
everything they can about new students” and to use student data to inform student 
placements, scheduling, and support (2006, pg. 245). Recent research on early predictors 
of high school success, described already under Domain E, shows the importance of data 
on students’ prior academic success, behavior, and attendance. 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ School review of the prior CST scores, grades, attendance and behavior records of 
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students entering the middle grades, and communication with elementary teachers. 

■ Provision of immediate, additional support for students who need it. 

■ The school’s coordination of student transitions into and out of the middle grades (e.g., 
orientation programs; mixing students from different feeder schools in class 
placements). 

■ District priorities for middle grade staff regarding the above. 

■ Consideration of the language needs of incoming EL students. 

Domain H — Teacher competencies, evaluation, and support 

Teacher preparation and professional growth in the middle grades have become 
increasingly important topics of policy discussion in California in recent years, especially 
in mathematics. For example, some have raised concerns about the supply of teachers 
holding single-subject credentials in mathematics in the state (California Council on 
Science and Technology and Center for the Future of Teaching and Learning, 2007), and 
about the preparation of middle grade Algebra I and other mathematics teachers (Guha, 
Shields, et al., 2008; California Commission on Teacher Credentialing, 2008b, 2009a, 
2009b; EdSource, 2009a). 

Credentialing and preparation to teach at the middle grade level is widely recognized to be 
insufficient without useful and ongoing opportunities for teachers’ professional 
development and growth, however. Practices and policies that support ongoing teacher 
collaboration and growth are central aspects of major reform and policy recommendations 
(e.g., National Staff Development Council, 2001; National Association of Secondary 
School Principals, 2006; CDE, 2008d; National League of Middle Schools, 2010), as well 
as the criteria used to identify middle grade “Schools to Watch” in California (National 
Forum to Accelerate Middle-Grades Reform and California Middle Grades Alliance, 
2009). These are also central to various comprehensive school reform models (e.g., Felner, 
Jackson, et al., 1997; Cooney & Bottoms, 2003; Balfanz, Ruby, et al., 2004; Maclver, 
Ruby, et al., 2007), frequently with particular focus on collaborative contexts for the 
development of standards-based practice. 

This study’s survey instruments inquire into such strands of policy and practice as: 

■ The principal’s perception of teacher skills and qualifications at his or her school. 

■ Teacher experience and education. 

■ Multiple- versus single-subject credentials, and other qualifications. 

■ Professional development and district/principal support. 

■ Teacher collaboration to discuss lessons and student work. 

■ Teacher involvement in decision-making. 

Domain 1-Principal leadership and competencies 

The era of standards-based reform has highlighted the importance of principals as leaders 
of the school improvement process, drivers of their schools’ vision and mission, and 
cultivators of human capital (e.g., Cotton, 2003; Leithwood, Steinbach, et al., 2002; 
Leithwood and Riehl, 2003; National Association of Secondary School Principals, 2006; 
Augustine, Gonzalez, et al., 2009; New Leaders for New Schools, 2009). As already 
described, the preceding domains highlight various aspects of principal leadership as they 
pertain to particular areas of practice and policy. These are also considered here as aspects 
of the principal’s role. 
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Ill addition to aspects of principal leadership considered in other domains, this study’s 
survey instruments inquire into such strands of the principal’s role as: 

■ The principal’s credentials, training, experience, and education. 

■ The principal’s demonstrated commitment to the middle grades. 

■ Whether the principal builds strong relationships with parents. 

■ Considerations that influence the principal’s academic priorities for the school. 

■ Evaluation of teachers and the district’s support for doing so. 

■ The district’s evaluation of the principal, review of student data with the principal, and 
support for professional growth and decision-making. 

Domain J -Superintendent leadership and district support 

School districts play an important role in guiding and supporting local capacity for 
continuous evaluation and instructional improvement (e.g., Massed, 2000; Fairman and 
Firestone, 2001; Knapp, Copland, and Talbert, 2003; McFaughlin and Talbert, 2003; 
Petrides, Nodine, et ah, 2005; Bottoms, Cooney, et ah, 2007; Augustine, Gonzalez, et ah, 
2009). The preceding domains highlighted such interrelated aspects of district leadership 
as common academic benchmarks and curricula, preferred credentials when hiring new 
middle grade teachers, support for ongoing professional development, evaluation of 
principals, and support of local capacity to use student data. These are also considered 
here as aspects of the district’s role. 

In addition, this study’s survey instruments inquire into such strands of policy and practice 
as: 



■ Effective district operations and capacity to support middle grade improvement. 

■ Considerations that influence the superintendent’s priorities for the district. 

■ The school board’s evaluation of the superintendent. 

■ Superintendent experience and education. 

Survey development 

The survey instruments developed for this study inquire into school and district practices 

and policies in the middle grades in California, based on the ten conceptual domains just 

discussed. 

The research team developed three survey instruments: 

■ A survey for school principals. This survey had the most items (447), concerning both 
school practices and policies and district expectations. In addition, principals indicated 
which state-adopted curriculum programs in Reading/Fanguage Arts and Mathematics 
were in use at their schools in 2008-09, and in what grades, using a provided list. 

■ A survey for English language arts and mathematics teachers. This survey was 
developed only for English language arts and mathematics teachers of record in grades 
6-8. Its 313 items included one large skip pattern, so that questions specific to each of 
the two subject areas were directed to appropriate teachers. The teacher survey focused 
primarily on schoolwide and classroom practices and policies, and asked a limited 
number of questions about district practices. 

■ A survey for district superintendents (or charter management organization leaders, 
when applicable). This survey, which had 186 items, inquired into district practices and 
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priorities for the middle grades. 

Several characteristics distinguish the survey instruments: 

■ The majority of items ask about whether a range of detailed, concrete, actionable 
practices and policies are in place at each school, and the extent to which respondents 
agree that they are in place. Schools may differ in the practices they report using, or in 
the intensity (typically on a five-point scale) to which they agree the practices are in 
place. 

■ The surveys were written as neutrally as possible. They provide a variety of acceptable 
response options for every area of inquiry — there are no “wrong” answers. 

■ Some common items bridge two or all three of the survey instruments — rephrased to be 
appropriate for different respondents (e.g., principals vs. teachers) — and other items 
ask respondents to report on their roles relative to one another. The goal of these items 
was to allow us to explore the “vertical alignment” of respondents’ answers: between 
the principal and teachers, and between the district and the school. 

■ Large portions of the surveys were developed in response to recent state and national 
debates about the middle grades and chart timely new ground. These included items 
pertaining to school practices and policies for placing middle grade students in English 
language arts and mathematics courses, student transitions into and out of the middle 
grades, and attention to student behavior and attendance problems. 

■ The surveys were written expressly to accommodate differences among the 303-school 
sample in terms of grade configuration, students’ socioeconomic status, and 
organization as a charter or traditional public school. 

Where appropriate and consistent with the research domains and its understanding of 
California’s middle grade policy contexts, the research team also adapted or were inspired 
by questions from pre-existing surveys by the Center for Social Organization of Schools at 
the Johns Hopkins University (2006; Center for Research on Elementary and Middle 
Schools, 1988), the Consortium on Chicago School Research at the University of Chicago 
(2007a, 2007b, 2007c), and the RAND Corporation (2006a, 2006b, 2006c), and by the 
research team's previous work (Williams, Kirst, Haertel, et al., 2005). 

A team from WestEd led by John Bosma, associate director of WestEd’s Evaluation 
Research Program, worked on contract with EdSource to ensure the quality of the survey 
instruments. Bosma and his staff provided expertise on such technical matters as the 
development of proper scales and helped refine the survey items to avoid “double- 
barreled” questions. The WestEd team also organized the teacher survey pilot sessions, 
formatted and produced the printed surveys, and delivered data files containing the survey 
responses. 

Determining the schools eligible to participate in the study 

The research team’s first task was to define a sample of schools that would be eligible to 
participate in the survey during the 2008-09 school year, based on earlier data on these 
schools available from the California Department of Education (CDE). 

Given the research question, the research team decided to use California’s School 
Characteristics Index (SCI) to select an initial sample of similar middle grade schools that 
would be eligible for recruitment for the study. (For further explanation of California’s 
SCI, see the box on the facing page.) The team hoped to recruit from a sample of 500 or 
more schools, which meant defining a fairly wide SCI percentile range (or “band”). 
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BACKGROUND 



California's School Characteristics Index (SCI) 

The research team used California’s School Characteristics Index (SCI) to identify the 
study’s two samples of middle grade schools. 

California education policymakers created the School Characteristics Index (SCI) to 
summarize multiple factors associated with student performance on state tests. The index 
permits comparisons of student achievement between schools with “similar characteristics.” 

The SCI is calculated slightly differently depending on whether a school is categorized for 
state accountability purposes as an elementary, middle, or high school. The SCI calculations 
used in this study considered school factors such as the following (see California 
Department of Education, 2008c): 



■ Socioeconomic indicators (average 
parent education, percent of students 
participating in free/reduced-price 
meals); 

■ Percent of students who are English 
learners (ELs) or have been 
redesignated as fluent English 
proficient (RFEP); 

■ Percent of students from different 
racial/ethnic groups; 

■ Percent of students with disabilities; 

■ Percent of students in the Gifted and 
Talented Education (GATE) program; 



■ Teacher credentials (percent of teachers 
who are fully credentialed, percent with 
emergency permits); 

■ Average class size; 

■ Percent of students first attending the 
school this year (i.e., student mobility); 

■ Whether the school operates a 
multitrack, year-round educational 
program; 

■ Percent of enrollment by grade span; 
and 

■ Percent of students in the Migrant 
Education Program. 



To meet this goal, the research team decided to identify a bimodal sample, with half of the 
schools serving predominantly low-income students and the other half serving 
predominantly middle-income students. A bimodal sample would also make it possible to 
analyze empirically whether practices and policies differentiating higher- from lower- 
performing schools differ between two such SCI bands, if at all. 

In addition, the research team decided to survey schools from all major middle grade 
configurations in California: K-8, 5/6-8, 7-8, and others. This decision was consistent 
with the team’s interest in surveying middle grade schools across California rather than 
only middle schools. The research team was also mindful that grade configuration is a 
topic of recurring interest for policymakers and researchers. 

Finally, the research team decided to include both traditional public schools and charter 
schools. 

Eligible schools also needed to meet the following criteria: 

■ Each school needed to be designated by California’s 2007 Base Academic Performance 
Index (API) as a middle school, or as an elementary school that the research team 
determined served students in both grades 7 and 8. K-12 schools and schools 
designated as high schools for state accountability purposes were excluded. 

■ Each school needed to have an SCI for the 2006-07 school year in order to make SCI 
band definitions possible. The research team recalculated a “middle school” SCI for 
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any middle grade school designated for state accountability purposes as an 
“elementary” school — as is often the case with K-8 schools — thus establishing a 
common SCI metric by which schools could be sorted and percentiles of the 
distribution calculated. This was done using the regression weights defining the 2006- 
07 middle school SCI. 

■ Schools that tested ten or fewer students on either the Grade 7 or 8 English Language 
Arts CST in 2007 were excluded. 

■ Alternative Schools Accountability Model (ASAM) schools — alternative schools of 
various kinds, mostly serving highly mobile, highly at-risk students — were also 
excluded. 

During the period from August to September 2008, the research team settled on the 20th- 
35th and the 7 0th-85th percentile bands of the SCI distribution as its definition for middle 
grade schools serving predominantly lower- and middle-income students, respectively. 
This yielded a total of 528 schools — 264 schools in each SCI band — that had served 
middle grade students during the academic year prior to the study for potential 
participation. (These are the same two groups of schools highlighted in Figure 1 on page 
9.) 

Accounting for schools later discovered to have closed or to no longer be serving students 
in grades 7 and 8, 516 middle grade schools operating during the 2008-09 academic year 
were eligible to participate in the study. These included: 

■ 255 middle grade schools serving predominantly lower-income students, located within 
the 20th-35th percentile band of the SCI; and 

■ 261 schools serving predominantly middle-income students, located within the 70th- 
85th percentile band of the SCI. 

Recruiting eligible schools for the study 

The research team set out to recruit as many of these schools to participate in the study as 
possible, in order to strengthen the statistical power of the analyses. The team also set an 
ambitious target that all schools would need to agree to return surveys for at least 60% of 
English language arts and 60% of mathematics teachers of record in grades 6-8, as well as 
a principal survey, in order to be eligible to participate in the study. (Special Education 
and resource teachers were not surveyed.) The research team also hoped to survey the 
superintendent of each participating school but did not make that a condition for school 
participation. 

EdSource took on the task of recruiting schools for the study because of its strong 
relationship with California school districts for more than 30 years. The process of 
recruiting schools and subsequently shipping and retrieving the survey instruments ran 
from November 2008 to July 2009. EdSource staff designed and provided oversight of this 
process. EdSource turned its small library into a campaign office and phone bank, hired a 
half-dozen active or former school board and PTA leaders to serve as the recruitment 
group, and developed a tracking and follow-up system. This group made numerous calls to 
district and school staff to encourage participation and followed up to encourage 
completion and return of the surveys. California’s economic climate and education 
funding cuts in 2008-09 made this a particular challenge. 

EdSource first asked superintendents for permission to contact eligible schools within 
their respective districts (or charter management organization, if appropriate). In some 
cases, this process involved a formal research review process. Out of 304 districts 
contacted, 227 gave permission; 77 declined permission, did not respond, or reported that 
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a school had closed or consolidated with another, thus eliminating 133 schools from 
consideration. Districts declining permission usually cited school duress over the 
uncertainty of California’s funding climate. 

For the remaining 395 schools, the EdSource recruitment group contacted each principal 
to explain the study and participation targets, and to invite participation. The recruitment 
group tracked all school responses by grade configuration and SCI percentile group. 

The time commitment for teachers was one of the greatest concerns expressed by the 
schools. Along with promising to do everything possible to minimize the time burden of 
the survey, EdSource offered a small incentive: a bookstore gift card of $20 for each 
school’s survey coordinator and principal, and every teacher who completed a survey. 
EdSource also offered a $100 cash donation to every school that met the participation 
targets (completed surveys from the principal and from 60% of both English language arts 
and mathematics teachers). 

The participating schools 

The recruitment process was largely complete by February 2009. Of the 395 schools 
contacted, 315 agreed to participate; however, only 303 ultimately followed up to provide 
the teacher and principal data included in the study. 

By the end of the survey retrieval process in spring 2009, the research team collected: 

■ 3,752 completed teacher surveys from the 303 schools. The average school-level 
teacher participation rate was 88%, and 142 schools returned 100% of their teacher 
surveys. 

■ A completed principal survey from each of the 303 schools. 

During the study year (2008-09), half of the 303 participating schools served grades 6-8 
(including a few schools with unusual configurations such as 5-8), 26% served grades 7- 
8, and 24% served grades K-8. 

Among the participating schools: 

■ 144 schools, including nine charter schools, were from the 20th-35th percentile 
SCI band group. During the 2008-09 school year, these schools were more likely than 
the California average to serve middle grade students who were socioeconomically 
disadvantaged, Hispanic, ELs, and/or whose parents had achieved no more than a high 
school diploma. Fifty-five (38%) of these schools were designated as being in Year 5 
of Program Improvement (PI) under the federal No Child Left Behind law in that year. 
The percentage of middle grade students who were African American was notably 
larger among the nine charter schools in this SCI band (26%) than among the non- 
charters (9%). 

■ 159 schools, including 19 charter schools, were from the 70th-85th percentile SCI 
band group. During the 2008-09 school year, these schools were more likely than the 
California average to serve middle grade students who were white and/or whose 
parents had completed some college or more. At the same time, however, nearly three 
in ten middle grade students in these schools were socioeconomically disadvantaged. 
Only one of these schools was designated as being in Year - 5 of PI in 2008-09, and 
charter and non-charter schools in this SCI band were very similar with respect to 
student demographics in the middle grades. 

The research team undertook statistical tests to determine the extent to which the 303 
schools are representative of the other schools in their respective SCI bands. For each SCI 
band, the team compared participating schools with non-participating schools with 
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respect to each of the following: 

■ Student demographics (socioeconomic disadvantage; English language proficiency; 
parent education; ethnicity); 

■ Middle grade cohort size; 

■ The average percentage of students (across all grades served) who were continuously 
enrolled during the school year; 

■ Mean school-level student achievement on the grades 6-8 English Language Arts 
CSTs, the grades 6 and 7 Mathematics CSTs, and General Mathematics and Algebra I 
CSTs taken by students in grade 8; and 

■ School grade configuration (K-8; 7-8; 6-8/other). 

(Full comparative data are available in Technical Appendix B.) 

The participating schools are representative of the SCI bands from which they were 
selected in virtually all these respects. There were no statistically significant differences 
(at the .05 level) between the participating and non-participating schools, in either SCI 
band, with respect to the ethnic backgrounds, socioeconomic disadvantage, or language 
status of the middle grade students they served, and virtually no differences with respect to 
parent education. There were also no differences with respect to the average percentage of 
students who were continuously enrolled during the school year or average middle grade 
cohort size, and virtually no differences with respect to school-level achievement on the 
seven CSTs considered (with the exception of a single English language arts outcome 
among schools in the 20th-35th percentile SCI band). 

There was a statistically significant difference between participating and non-participating 
schools — slightly more pronounced in the 70th-85th percentile SCI band — with respect to 
grade configuration, however. K-8 schools were somewhat under-represented in the 
study, and 7-8 and 6-8 schools were somewhat over-represented. 



An extraordinary response rate: Why schools participated in the study 

The research team solicited feedback from participating middle grade principals on why their 
schools participated in the study during an economically difficult school year, and how the 
research process and survey could be improved in the future. 

More than 1 50 principals returned a “One Minute Feedback” form by fax. The form provided 
the principals with 12 possible reasons why their schools might have committed the time to 
complete the teacher and principal surveys. Responding principals placed a check mark 
next to all reasons that applied to them, and placed a star next to their three strongest 
reasons for participating. 

The principals most frequently cited the following as being among their strongest reasons for 
participating: 

■ The research focus was on improving middle grade student achievement. 

■ The principal had confidence that a study/survey done by EdSource would be 
meaningful. 

■ The gift card for each participating teacher and principal made a difference. 

■ The survey focus was on concrete, actionable school and district practices. 

This and other feedback provided by the principals will inform future EdSource research. 
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Figure 3: Distribution of participating schools by region in California 
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The participating districts 

The 303 schools in this study are governed by 195 different California school districts plus 
six charter management organizations (CMOs). Most of the schools are located in unified 
school districts that serve grades K-12, about a third operate in elementary districts, and 
1 1 schools were located in high school districts. 

In most cases, a given district was home to only one school that participated in the study. 
However, 59 districts had two or more schools among the final research sample; four of 
these districts were home to more than five participating schools. 

After the surveys for principals and teachers were mailed to schools — between January 
and March 2009 — the research team finalized the superintendent survey. From March to 
June 2009, the EdSource recruitment group contacted district superintendents and CMO 
leaders who had at least one participating school to ask them to complete the 
superintendent survey. (Superintendent-principals who had already completed the 
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principal survey — a total of 1 8 — were not asked to also complete the superintendent 
survey.) 

Out of 177 district superintendents and seven CMO leaders to whom EdSource mailed 
surveys, 152 district superintendents and five CMO leaders completed them. These 
leaders represented 244 of the 303 schools in the sample. Of the responding 
superintendents, 49 lead elementary districts, 96 lead unified districts, and seven lead high 
school districts. (See Figure 4 below.) 



Figure 4: The majority of those who completed a superintendent survey were from 
unified (K-12) school districts 
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Analyses conducted 

(For full technical details of the analyses conducted for this study, see Technical Appendix 
A.) 

After all survey responses for the 303 participating schools had been collected, the 
research team set about the task of constructing independent variables in order to represent 
each school’s reported practices and policies. 

First, the research team categorized survey items as instances of one of the ten research 
domains described above on an a priori basis, guided by the same theory and practice 
considerations that produced the research domains and survey instruments. The team then 
grouped individual survey items further into subdomains, with each subdomain 
constructed to measure some aspect of the broader practice domain into which it was 
placed. Teacher survey items were grouped into teacher subdomains, principal items into 
principal subdomains, and superintendent items into superintendent subdomains. The team 
made every effort to group multiple items pertaining to a common practice into a single 
subdomain; however, some subdomains included only one survey item. 

In addition, the research team placed each subdomain in only one of the ten larger practice 
domains, so as to minimize overlap between them. The only exceptions were subdomains 
that pertained to either the principal’s or superintendent’s roles with respect to a domain- 
specific policy or practice. For example, a subdomain pertaining to the principal’s use of 
data might appear in both Domain E (“Extensive use of data to improve instruction and 
student learning”) and Domain I (“Principal leadership and competencies”). 

The research team then performed a statistical analysis to measure the reliability of the 
groups of items assigned to each subdomain. When necessary, the team made revisions to 
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ensure that each subdomain both substantively represented and statistically measured a 
single underlying dimension of variation in schooling policy or practice. 

The final subdomains became school-level, independent variables measuring the presence 
and/or intensity of different practices and policies at each school. Typically, the research 
team created these variables by averaging across the items contained in the subdomain. In 
addition, teacher survey responses were averaged up to the school level, with these school 
averages representing all teachers or subsets of teachers (e.g., by grade or subject) 
appropriate for different outcome variables. The research team also defined relevant pools 
of subdomains for use in building statistical models for each CST outcome. For instance, a 
subdomain pertaining to instructional practices in mathematics would not be considered in 
predicting English language arts outcomes. 

As noted earlier, seven California Standards Tests (CSTs) in English language arts and 
mathematics — taken by roughly 204,000 middle grade students among the participating 
schools — provided the dependent variables for this study. The research team analyzed the 
relationship of school practices and policies with school-level outcomes on these CSTs, 
after controlling for student demographics and other school characteristics. The control 
variables used in these analyses included school-level measures of student demographics 
(i.e., student ethnicity, socioeconomic status, mobility, and parent education); average 
middle grade cohort size; school grade configuration; and — in the case of mathematics 
outcomes for grades 7 and 8 — various school-level measures of student participation in 
Algebra I. (See Technical Appendix A for a complete list of control variables.) 

Two sets of analyses were conducted: 

■ Cross-sectional regression analyses measured the power of each of the ten domains, 
and the subdomains within them, in explaining variation in the school-level CST 
outcomes in 2008-09. These analyses used school-level mean scale scores as the 
outcomes of interest. 

■ Longitudinal regression analyses measured the power of each of the ten domains, and 
the subdomains within them, in explaining variation in school-level mean student 
growth, by controlling for prior student achievement. These analyses used a data file 
with test scores spanning four years for the students in the sample to construct a 2009 
“predicted” mean test score for the students in each school, based upon their prior 
achievement. These analyses identified which school and district practices were most 
strongly associated with student achievement beyond the predicted levels, as measured 
by the estimated portion of school-level CST scores for 2009 that could not be 
explained by prior-year scores. (The research team requested a set of restricted-use, 
student-level data files from the California Department of Education to permit the 
construction of these growth outcome variables. See Technical Appendix A for a 
detailed description of how the research team constructed these longitudinal outcomes.) 

These analyses were designed to determine which reported schooling practices were most 
positively correlated with higher school-level achievement outcomes, other things being 
equal. The research team ran domain-specific regression analyses for each CST outcome 
(defined both cross-sectionally and longitudinally), doing so for the full (or “pooled”) 
sample of schools and for each SCI band. In addition, the team checked all subdomain 
variables in the final models to determine whether any statistically significant 
relationships between practices and achievement were limited to only one or the other SCI 
band. The team also analyzed whether any differences between the two SCI bands were 



1 To reprise, these included grade-level outcomes on the English Language Arts CSTs for each of grades 6, 7, 
and 8; grade-level outcomes on the Mathematics CSTs for each of grades 6 and 7; and 8th graders’ outcomes 
on the General Mathematics and Algebra I CSTs. 
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statistically significant. 

The Findings section of this report describes in detail those subdomains under each 
domain for which a positive and statistically significant relationship with achievement was 
found, after accounting for demographic and school variables and other domain-specific 
practices. (See Technical Appendix C for a summary of the decision rules that the research 
team employed in making judgments about which regression outputs constituted 
reportable findings.) 

The findings related to practices reported by principals and teachers are based on 
regression models using only the principal and teacher subdomains, using the largest 
school samples possible. The findings related to practices reported by superintendents are 
based on augmented regression models that also included superintendent subdomains, 
using the subsets of schools for which superintendent survey responses were available. 

(See Technical Appendix A for a detailed explanation of how superintendent subdomains 
were considered in the regression analyses.) 

To further investigate the findings, the research team also determined the strength of the 
ten study domains in explaining achievement outcomes relative to one another. The final 
step, after building models for each domain separately, was to compare the domains in 
terms of their explanatory power for each outcome, and to test whether differences in their 
explanatory power were statistically significant. The substantive results of these 
comparisons are reported at the outset of the Findings section. (See Technical Appendix A 
for a detailed description of how the research team carried out these comparisons.) 

Limitations of the study 

Before presenting the study findings, it is fitting to note key limitations of the 
methodology and analysis just discussed. Every type of study has strengths and 
weaknesses, and this study is no different. 

■ Because the survey instruments covered a broad range of middle grade policies and 
practices, they could not delve deeply or narrowly into any one particular domain of 
effective-schools practices. 

■ Surveys are by nature “self reports” by the educators who complete them. The research 
team did not conduct school or district site visits to observe and verify implementation 
of the practices reported by responding superintendents, principals, or teachers. The 
size of the 303 -school sample and the design of the survey administration — which 
included high teacher-participation rates at each school and subsequent consistency 
checks among teacher responses — add to the validity of the survey responses on which 
the analyses are based, however. 

■ Although the analyses found correlations between some practices and student 
outcomes on the California Standards Tests (CSTs), the estimated relationships should 
by no means be interpreted as causal. No non-experimental study of this kind can claim 
to identify causal factors between schooling practices and achievement outcomes. 

■ This study is not an evaluation of any particular middle grade program or philosophy, 
nor is it a study of middle grade pedagogy. 

■ The outcome measures used in this study are limited to scores on standards -based state 
tests in English language arts and mathematics taken by middle grade students. As a 
result, the study’s characterization of school performance reflects the limitations of the 
tests in these two particular subject areas, which cannot fully capture a school’s 
effectiveness. This study does not consider outcomes in other subject areas, nor does it 
consider outcomes pertaining to other important aspects of middle grade school 
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effectiveness, such as student engagement, attendance, or health. 

■ Finally, this study considers only CST outcomes for schools as a whole. Special 
Education teachers were not surveyed, and the outcomes studied do not include the 
results of assessments taken by many students who receive Special Education services 
(the California Modified Assessments or the California Alternate Performance 
Assessment). And although some survey items (and some subdomains) pertain to 
practices specific to the middle grade education of English learner (EL) students, this 
study does not examine the CST outcomes of EL students as a separate student group, 
nor does it investigate the CST outcomes of any other student group reported 
separately under the federal No Child Left Behind law. 
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IV. Findings 

This section presents the study findings in detail for each of the ten research domains. As 
explained above, the research team grouped survey items measuring similar themes or 
practices into subdomains within the ten larger domains, with superintendent, principal, 
and teacher items being grouped separately. Each subdomain exemplifies some aspect of 
the domain in which it is located. 

The cross-sectional and longitudinal regression analyses conducted reveal both: 

■ Which domains most strongly predict higher school-level CST outcomes, after 
controlling for student demographics and other factors; and 

■ Which of the many subdomains within each domain correlate positively with CST 
outcomes, at the .05 level of statistical significance. 

The ten research domains fell into three “groupings” with respect to their relative 
predictive strength in differentiating higher-performing middle grade schools in the 
sample. (See the Technical Appendix C for the respective effect sizes for each domain 
across the multiple regression analyses.) 

■ The domain with the greatest predictive strength across most of the cross-sectional and 
longitudinal analyses was Domain B, “An intense, school-wide focus on improving 
academic outcomes.” 

■ Six domains consistently followed Domain B in terms of predictive strength. Relative 
to one another, their predictive strength varied depending on the analysis, and there 
were relatively few statistically significant differences among them. These domains 
pertain to standards-based instruction and curricula; extensive use of data; proactive 
academic interventions; teacher competencies; principal leadership; and superintendent 
leadership. 

■ Three domains, although they did differentiate schools with higher CST scores and 
higher gains in CST scores, did so with less predictive strength than the other seven 
domains in almost every analysis conducted. These three domains pertain to school 
environment; the organization of time and instruction; and attention to student 
transitions. 

Although identifying the relative predictive strength of these three domain groupings is an 
important finding, equally important — if not more so — are the specific district- and 
school-level practices under each domain that distinguished higher-performing middle 
grade schools in the cross-sectional and longitudinal analyses. 

These particular findings are presented below, by domain. For each domain, the practices 
and policies reported as differentiating higher-performing schools are based on 
superintendent, principal, and/or teacher subdomains that correlated positively with one or 
more CST outcomes at the .05 level of statistical significance, among the cross-sectional 
and/or longitudinal analyses. (See Technical Appendix C for the decision rules the 
research team used in making judgments about reportable findings, as well as the actual 
survey items included in each reported subdomain.) 

It is important to note that higher-performing schools are often set apart not by the mere 
fact that a particular practice is reported, but by how consistently and intensely (typically 
on a scale of 1 to 5) each school’s principal and teachers report implementing the practice, 
or by how strongly the superintendent reports the practice or policy. 
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Overall, there was substantial overlap between the cross-sectional and longitudinal 
analyses with respect to findings. (See Figure 5 below.) Of the 1 19 subdomains described 
in the following pages, 56 of them differentiated schools with higher achievement in both 
sets of analyses. Overlap varied among domains and in some cases was extensive. One 
interesting case of comparatively less overlap — Domain D: Coherent and aligned 
standards-based instruction and curricula — is described later. 

Even when there were differences in the specific subdomains highlighted by the cross- 
sectional and longitudinal analyses, respectively, there were not substantive conceptual 
differences in the types of practices and policies these subdomains highlighted as 
differentiating higher-achieving schools. Rather, these practices and policies were 
complementary. 



Figure 5: There was substantial overlap between the longitudinal and cross-sectional 
analyses with respect to subdomain findings 
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One domain consistently had the greatest predictive strength 

■ Domain B: An intense, school-wide focus on improving academic outcomes 

In analyzing schools’ and districts’ reported practices, Domain B was consistently the 
strongest predictor of school-level CST outcomes. When considered in terms of all 
English language arts and mathematics CST outcomes combined, this domain was the 
single most influential differentiator of higher -performing schools in both the cross- 
sectional and longitudinal analyses. 

This was true for the total (or “pooled”) sample of schools, and for both the lower-income 
and middle-income subsamples of schools. Domain B also consistently had a higher 
impact than all other domains on CST outcomes in mathematics, whether in the cross- 
sectional or the longitudinal analyses. 

The subdomains under this domain that differentiated higher-performing schools each, in 
some way, emphasize the importance of actors at all levels — the district, the principal, 
teachers, parents, and students — setting, taking responsibility for, and holding one another 
accountable for meeting clear" educational goals. 

Certain of these goals — such as meeting or exceeding state and federal accountability 
targets — clarify how such schools report acting in response to public accountability 
measures. Others focus on the middle grades as essential for students’ futures. For 
example, middle grade schools in which the principal and/or teachers reported that 
curriculum and instruction are designed to prepare students for a rigorous high school 
curriculum tended to have better school-level achievement across many CST outcomes in 
both the cross-sectional and longitudinal analyses. 
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FINDINGS 



Domain B-An intense, school-wide focus on improving academic outcomes 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

A curriculum designed to prepare students for high school: 

■ Principal and teachers report that the school’s curriculum and instruction are designed to 
prepare students for a rigorous high school curriculum. 

Setting and prioritizing measurable objectives for student achievement: 

■ Superintendent reports that she/he expects middle grade principals and staff to prioritize 
improving student achievement (for subgroups and regardless of proficiency level), and 
that the district sets measurable objectives for improving CST scores across all 
performance levels and by grade level and subject area. 

■ Principal reports that the district emphasizes improving student achievement across all 
CST levels, closing CST subgroup achievement gaps, and getting as many students to 
proficient as possible. 

■ Principal and teachers report that the school emphasizes improving achievement across all 
CST performance levels and sets measurable goals for CST scores by grade and subject 
matter. 

■ Principal reports that the school sets measurable goals for improving district benchmark 
test scores. 

■ Principal reports that English language arts and mathematics teachers take responsibility 
for improving their students’ achievement; and that grade and subject-matter teams set 
goals for student achievement. 

The priority of state and federal accountability targets: 

■ Teachers report that meeting and exceeding AYP (Adequate Yearly Progress) and API 
(Academic Performance Index) targets are school priorities. 

■ Principal reports that she/he places a priority on meeting AYP subgroup targets, and 
meeting and exceeding API growth targets. 

Principal expectations: 

■ Principal reports that she/he regularly communicates to faculty the importance of high 
expectations for the achievement of all students. 

Uninterrupted instructional time: 

■ Teachers report that instructional time in their classrooms is protected from unnecessary 
interruptions. 

Student and parent responsibility for student learning: 

■ Teachers report that they frequently communicate with parents about middle grade 
academic standards and students’ progress, and provide ways for parents to support their 
students’ academic achievement. 

■ Principal reports that the school regularly communicates to students the importance of 
middle grade achievement to their future and of taking responsibility for their learning, and 
communicates to parents the importance of their role in setting high expectations for 
student academic success. 

■ Principal reports that the school has requirements or contracts for parent participation. 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Teachers report that the district communicates high expectations that schools will meet or 
exceed state and federal accountability targets. 

■ Principal reports that the school uses student progress and achievement data as part of 
teacher evaluations. 
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The six, next-strongest domains varied in their relative predictive strength, depending on the 
analysis 

Next in terms of overall predictive strength were six domains that varied, depending on the 
analysis, in the power with which they differentiated higher-performing schools relative to 
one another. They are, in no particular order: 

■ Domain D: Coherent and aligned standards-based instruction and curricula. 

■ Domain E: Extensive use of data to improve instruction and student learning. 

■ Domain F: Early and proactive academic interventions. 

■ Domain H: Teacher competencies, evaluation, and support. 

■ Domain I: Principal leadership and competencies. 

■ Domain J: Superintendent leadership and district support. 

The relative predictive strength of these domains varied depending on: 

■ The type of analysis conducted (i.e., cross-sectional vs. longitudinal); 

■ Whether superintendent survey responses were included (which reduced the sample 
size); and 

■ The CSTs considered (i.e., all English language arts vs. all mathematics vs. both 
subject areas combined). 

In addition to their varying “order” — and again depending on the analysis — there were 
frequently very few statistically significant differences among these six domains with 
respect to their relative predictive strength. See Technical Appendix C for the respective 
effect sizes of these domains across the multiple analyses. 

Domain D-Coherentand aligned standards-based instruction and curricula 

The subdomains that differentiated higher-performing schools under this domain highlight 
schools’ ongoing evaluation of their instructional improvement plans and the concrete 
actions they report taking to align instruction with state standards and ensure curricular 
coherence among teachers. Regarding the latter, for example, schools in which teachers 
reported collaborating on curriculum scope and common benchmark assessments, and 
closely aligning instruction with state standards — such as by focusing on key standards or 
breaking down prerequisite skills — had better school-level achievement on multiple CST 
outcomes, across both the cross-sectional and longitudinal analyses. 

Districts’ reported emphasis on attention to the placement of English learners in English 
language arts and mathematics classes, and the school’s reported consideration of a range 
of factors when placing students into general mathematics classes in grades 7 and 8, were 
also associated with higher achievement. In addition, principals’ reports of a structured 
program to promote literacy across the grade 8 curriculum were associated with higher 
school-level outcomes in 8th grade English language arts in the longitudinal analysis. 

Interestingly, reported practices and policies related to curriculum adoption and 
expectations about how frequently the curriculum should be used by teachers emerged as 
much stronger findings in the longitudinal analyses than in the cross-sectional. Schools 
where respondents reported a stronger district role in driving curriculum adoption and/or 
the alignment of instruction with standards, and/or where principals reported expecting 
more frequent use of adopted curricula (e.g., daily) by English language arts and 
mathematics teachers, tended to have higher school-level outcomes after controlling for 
prior student achievement. 
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FINDINGS 



Domain D-Coherentand aligned standards-based instruction and curricula 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

Planning for instructional improvement: 

■ Principal reports that the school has well-defined plans for instructional improvement and 
assesses their effectiveness regularly. 

■ Teachers report that the school assesses the effectiveness of its instructional improvement 
plans on an ongoing basis. 

Administrative expectations for curricula and instruction: 

■ Principal reports that the district leads decisions about the school’s 6th— 8th grade 
curriculum program adoption in English language arts. 

■ Principal reports that she/he expects the school’s teachers to use the school’s adopted 
English language arts and mathematics curriculum programs frequently. 

■ Principal reports that the school’s English language arts and mathematics instruction is 
closely guided by state academic standards and state-adopted curriculum programs, takes 
into account English Language Development standards, and emphasizes key standards in 
each grade and core subject. 

■ Teachers report that the district communicates high expectations that instruction will 
closely align with state standards. 

Teacher engagement with standards and curricula: 

■ English language arts and mathematics teachers each report that they closely align 
instruction with the California academic content standards and CSTs, emphasize key 
standards, and work together to break down prerequisite skills. 

■ Teachers report that they collaborate on curriculum pacing, common benchmark 
assessments, and instruction. 

■ English language arts teachers report that they modify or augment the adopted English 
language arts curriculum with additional materials to meet the needs of students. 

■ Mathematics teachers report that they augment the adopted mathematics curriculum. 

Attention to student placement: 

■ Superintendent reports that the district expects middle grade schools to consider a range 
of factors when placing non-newcomer English learners in English language arts and 
mathematics classes. 

■ Mathematics teachers report that the school considers a range of factors in placing 
students into general mathematics classes in grades 7 and 8. 

Student literacy: 

■ Principal reports that the school has a structured program to promote literacy across the 
grade 8 curriculum. 

■ English language arts teachers report that they hold students accountable for reading 
outside of class and teach strategies for writing. 

The study also found, for schools in the 70th-85th percentile of the SCI: 

■ Superintendent reports that the district leads decisions about schools’ 6th— 8th grade 
curriculum program adoptions in English language arts and mathematics. 

■ Principal reports that the school has explicit written criteria for placing students in 
mathematics classes, department chairs revise math placements for academic 
appropriateness, and the administrative team reviews placements to ensure wide access 
to a rigorous math curriculum. 



©2010 EdSource 



37 




Domain E-Extensive use of data to improve instruction and student learning 

Taken together, the findings in this domain indicate a strong, schoolwide culture focused 
on student outcomes, such that a school uses data to inform decisions about additional 
academic support and student placement, and to set goals for meeting the needs of 
individuals and student subgroups. 

The subdomains that differentiated higher-performing schools under this domain also 
highlight how actors at multiple levels — district, principal, and teachers — report taking 
action to support the ongoing improvement of instruction through assessment and the 
analysis of student data. For example, schools whose teachers reported that their districts 
provide timely data and a user-friendly information system tended to have higher 8th 
grade mathematics outcomes in both the cross-sectional and longitudinal analyses. 
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FINDINGS 



Domain E-Extensive use of data to improve instruction and student learning 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

District support for local data capacity: 

■ Teachers report that the district supports their data analysis by providing schools with 
timely CST data and a user-friendly information system, and emphasizing regular teacher 
use of benchmark and/or diagnostic assessments. 

■ Principal reports that the district provides schools with timely CST, CELDT (California 
English Language Development Test), and district benchmark data; provides a computer- 
based data system and adequate staff training; and uses achievement data to improve and 
recognize teacher practice. 

Principal use of data: 

■ Principal reports using assessment data extensively and in multiple ways to improve 
student learning and teacher practice. 

■ Principal reports that she/he meets with teachers individually, by grade, and by subject to 
review CST results (including subgroups), and meets with the entire school to review 
schoolwide CST scores. 

Teacher access to CST data: 

■ Teachers report that they receive individual CST data for all students they teach, as well as 
data disaggregated by skill and subgroup and summarized across grade levels. 

Data-informed instruction, planning, and reflection on practice: 

■ Principal reports that teachers regularly monitor student grades and class test scores to 
rapidly report student intervention needs, discuss and use data to evaluate the 
achievement of different student groups, and set measurable student goals. 

■ Teachers report that they use assessment data to evaluate individual student achievement, 
set measurable goals, help students see progress, correct gaps in instruction, and identify 
effective instructional practices. 

■ Teachers report that they use assessment data to evaluate student achievement and set 
measurable goals by subgroup. 

■ Teachers report that the school uses CST scores for placement, promotion, and/or 
intervention. 

■ Teachers report that they frequently administer benchmark, diagnostic, and classroom- 
based assessments of student learning to inform their teaching. 

■ Superintendent reports that the district gives middle grade staff discretion over developing, 
determining the need for, and analyzing the results of diagnostic assessments. 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Principal reports that she/he meets with teachers individually, by grade, and by subject to 
review benchmark test results, and meets with the entire school staff to review schoolwide 
benchmark scores. 

The study also found, for schools in the 70th-85th percentile of the SCI: 

■ Teachers report that attendance and/or behavior teams meet to review student data and 
devise solutions. 
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Domain F-Early and proactive academic interventions 

The subdomains that differentiated higher-performing schools under this domain highlight 
a range of reported practices and policies for providing additional academic support to 
different student groups when needed and the positive roles to be played by: 

■ Districts, such as setting priorities for addressing the needs of students who are two or 
more years behind grade level; 

■ Principals, such as ensuring common planning time for English language arts and 
mathematics teachers to meet with intervention teachers to coordinate instruction; and 

■ Subject-area teachers, such as acting on the principal’s expectation for coordination 
with intervention teachers and providing different forms of support in the classroom. 

Also tending to have higher achievement in both the cross-sectional and longitudinal 
analyses were schools that reported offering required interventions, voluntary support, 
and/or individualized intervention plans developed by staff and parents for: 

■ Students at risk of failure in the current year; 

■ Students identified for intensive intervention (i.e., who are two or more years behind 
grade level); and/or 

■ English learner (EL) students. 



40 



Initial Research Report — Gaining Ground in the Middle Grades 



FINDINGS 



Domain F-Early and proactive academic interventions 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

District emphasis on early identification and student support: 

■ Principal reports that the district emphasizes early identification of students needing 
academic support and addresses the needs of students two or more years behind grade 
level. 

■ Teachers report that the district ensures that policies and resources are in place to serve 
English learner (EL) students. 

■ Principal reports that the district emphasizes the importance of English language 
development and subject-matter learning by EL students and addresses the needs of EL 
students who are new immigrants. 

Collaboration between subject-area and intervention teachers: 

■ Principal reports that she/he ensures common planning time is available for English 
language arts and mathematics teachers to meet with intervention teachers to coordinate 
instruction. 

■ Teachers report that subject-area teachers meet with intervention teachers to coordinate 
instruction. 

Required intervention strategies: 

■ Principal and teachers report that the school employs a range of required intervention 
strategies for students at risk of failure in the current year; principal reports the same for 
students identified for intensive intervention (i.e., who are two or more years behind grade 
level). 

Voluntary academic support: 

■ Principal reports that the school provides voluntary academic support (such as during non- 
classroom time, AVID, online tutorials) for students at risk of failure in the current year; 
teachers report the same for students identified for intensive intervention. 

Individualized intervention plans developed by staff and parents: 

■ Principal and teachers report that school staff and parents meet to develop intervention 
plans for individual students at risk of failure and students identified for intensive 
intervention; principal reports the same for individual EL students. 

Differentiated instruction and flexible grouping: 

■ Teachers report using differentiated instruction and flexible grouping in the classroom. 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Teachers report that principal assigns teachers to ensure students with the greatest need 
are served well. 

The study also found, for schools in the 70th-85th percentile of the SCI: 

■ Teachers report that they pre-teach lessons (e.g., lead some students through a lesson the 
day before). 
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Domain H — Teacher competencies, evaluation, and support 

The subdomains that differentiated higher-performing schools under this domain — across 
both the cross-sectional and longitudinal analyses — draw attention to the importance of 
teacher evaluation and professional development. In particular, schools where teachers 
reported strongly that the principal provides meaningful evaluations of their practice were 
more likely to have better school-level achievement outcomes, as were schools where 
teachers and/or principals reported that the district provides useful professional 
development for teachers. 

Schools in which principals reported that their schools’ respective mathematics and 
English language arts teachers had a range of characteristics associated with effective 
teachers also tended to do better. Teachers’ preference for teaching at the middle grade 
level was also a predictor for some achievement outcomes. 

Many subdomains already discussed with respect to other domains describe more fully the 
range of reported teacher practices that differentiated higher-performing schools in the 
various analyses, other things being equal. Taken together, these indicate a culture of 
collective and individual actions focused on effective instruction and improved student 
outcomes. 

This study could determine no clear relationship between teacher credentialing practices 
and policies and CST outcomes, however. 



42 



Initial Research Report — Gaining Ground in the Middle Grades 



FINDINGS 



Domain H — Teacher competencies, evaluation, and support 

Other things being equal, stronger agreement about the following was associated with higher 
school performance 

Useful professional development provided by the district: 

■ Principal and teachers report that the district provides useful professional development for 
teachers. 

Meaningful teacher evaluation and other principal support for teaching: 

■ Teachers report that the principal: 

• Understands and acknowledges excellent teaching. 

• Arranges for evaluation of teaching skills by teacher leaders. 

• Ensures that evaluations of teaching are substantive and meaningful. 

• Ensures that teachers receive effective professional development to improve 
instruction. 

• Builds strong relationships with teachers and staff. 

Collaboration to evaluate and discuss practice: 

■ Teachers report that they collaborate to evaluate lessons and discuss student work. 

Principal evaluation of teacher characteristics: 

■ Principal reports that a high proportion of the school’s mathematics and English language 
arts teachers have such characteristics as: 

• The ability to use student assessment data to improve learning. 

• Knowledge of California’s state standards. 

• Fitting well into the school culture. 

• The ability to raise student achievement. 

• Strong subject-area knowledge. 

• The ability to map curriculum standards to instruction. 

• Being likely to remain in the teaching field. 

• Enjoying teaching at the middle grade level. 

• Understanding adolescent developmental issues. 

• Having taught at the school last year. 

• The ability to collaborate effectively with peers. 

• The ability to make personal connections with students. 

• Being well prepared by their teacher credential programs (new teachers). 

• Having expertise in working with English learner (EL) students. 

Teacher preference for teaching in the middle grades: 

■ Teachers report that middle grades are their current first choice of teaching assignment. 

The study also found, for schools in the 70th-85th percentile of the SCI: 

■ Principal reports that teachers are surveyed to determine ranked preference of 
subject/grade-level assignments. 
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Domain 1-Principal leadership and competencies 

Domain I indicates the importance of strong principal leadership in driving and 
orchestrating a schoolwide focus on improving student outcomes. 

Many of the subdomains that differentiated higher -performing schools under this 
domain — such as those pertaining to the principal’s use of data, expectations about 
frequency of curriculum use by teachers, and so forth — reprise topics already discussed 
under other domains. This is by design. As described earlier, subdomains pertaining to the 
principal’s reported role in using student data within the school, for example, appeared in 
two sets of regressions: those pertaining to Domain E (on extensive use of data) and here 
in Domain I (as an aspect of principal leadership). Thus, many of the findings reported 
below reinforce findings from other domains, but within a different conceptual context. 
(Such relationships between domains are specified in the box on the facing page.) 

Domain I also highlights other aspects of reported policy and practice related to the 
principal’s role that differentiated higher-performing schools. These include aspects of the 
principal’s relationship with the school district, such as principal evaluations that consider 
middle grade student achievement. These also include the principal’s training with respect 
to the instruction of English learner students; ensuring a clean and safe school 
environment; and communicating high expectations for student achievement and holding 
teachers accountable for common planning time to this end. 



44 



Initial Research Report — Gaining Ground in the Middle Grades 



FINDINGS 



Domain 1-Principal leadership and competencies 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

Subdomain findings unique to this domain: 

■ Superintendent reports that she/he evaluates middle grade principals on the effectiveness 
of academic interventions, improving student performance on district benchmarks and 
CSTs, improving EL students’ English proficiency, and exceeding state and federal 
accountability targets. 

■ Principal reports that her/his priorities for schoolwide improvement are influenced by 
expectations from superintendent/school board, CST scores, and state accountability 
measures. 

■ Principal reports that her/his priorities for schoolwide improvement are influenced by 
recommendations from an external provider (if in Program Improvement under the No 
Child Left Behind law). 

■ Teachers report that the principal regularly communicates the importance of high 
expectations for student achievement and holds teachers accountable for common 
planning time to focus on student achievement. 

■ Teachers report that the principal ensures a clean, safe, and disciplined school 
environment. 

■ Principal reports that she/he has been trained in the evaluation of instruction for EL 
students, understands the principles of EL second language acquisition, and is comfortable 
conversing in a language other than English. 

Subdomain findings also reported in other domains: 

■ Principal reports that she/he expects the school’s English language arts and mathematics 
teachers to use the school’s adopted curriculum programs frequently. ( Domain D) 

■ Principal reports using assessment data extensively and in multiple ways to improve 
student learning and teacher practice. ( Domain E) 

■ Principal reports that she/he meets with teachers individually, by grade, and by subject to 
review CST results (including subgroups) and/or benchmark test results, and meets with 
the entire school staff to review schoolwide CST and/or benchmark scores. ( Domain E) 

■ Principal reports that she/he ensures common planning time is available for English 
language arts and mathematics teachers to meet with intervention teachers to coordinate 
instruction. (Domain F) 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Teachers report that the principal assigns teachers to ensure students with the greatest 
need are served well. ( Domain F) 

■ Principal reports that she/he meets with district administrators to review the school’s CST 
scores, benchmark test scores, AYP and API progress, and grades and/or grading policy. 

( Domain J) 

■ Principal reports district support if she/he wants to replace the school administrative and/or 
instructional leadership team. ( Finding unique to this domain) 
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Domain J -Superintendent leadership and district support 

Domain J indicates that academic achievement in the middle grades can be supported 
when district leaders set a clear academic vision, follow through with accountability for 
student outcomes, and provide comprehensive resources to the middle grades (e.g., 
professional development, experienced district staff, data support). 

As with Domain I, many of the subdomains that differentiated higher-performing schools 
under this domain — such as those pertaining to district support for local use of data, useful 
professional development for teachers, and emphases on improving middle grade student 
achievement and early identification of students needing support — reprise and reinforce 
topics already discussed under other domains. (Such relationships between domains are 
specified in the box on the facing page.) 

Domain J provides additional context for understanding how district policies and practices 
might foster conditions that support higher middle grade student achievement. Subdomain 
findings unique to this domain pertain to such reported practices as evaluation of the 
district superintendent by the school board based in part on middle grade outcomes; 
consideration of student outcomes when setting district priorities for the middle grades; 
and providing the financial and human resources necessary for middle grade schools to 
meet their goals. 
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Domain J -Superintendent leadership and district support 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

Subdomain findings unique to this domain: 

■ Superintendent reports that her/his evaluation is based in part on middle grade academic 
outcomes. 

■ Superintendent reports that the district’s priorities for middle grade improvement are 
influenced by analysis of student CST scores and progress on other measures, and by 
published statewide ranking of schools. 

■ Superintendent reports that the district’s priorities for middle grade improvements are 
influenced by categorical funding requirements and private grant expectations. 

■ Superintendent reports that the district functions well across many dimensions (e.g., union, 
board, staffing, finance). 

■ Teachers report that the district provides the financial and human resources necessary for 
the school to meet its goals and that district staff visits the school at least once a year. 

Subdomain findings also reported in other domains: 

■ Principal reports that the district emphasizes improving student achievement across all 
CST levels, closing CST subgroup achievement gaps, and getting as many students to 
proficient as possible. ( Domain B) 

■ Principal reports that the district leads decisions about the school’s 6th— 8th grade 
curriculum program adoption in English language arts. ( Domain D ) 

■ Teachers report that the district communicates high expectations that instruction will 
closely align with state standards. ( Domain D) 

■ Principal reports that the district provides schools with timely CST, CELDT, and district 
benchmark data; provides a computer-based data system and adequate staff training; and 
uses achievement data to improve and recognize teacher practice. ( Domain E) 

■ Teachers report that the district supports their data analysis by providing schools with 
timely CST data and a user-friendly information system, and emphasizing regular teacher 
use of benchmark and/or diagnostic assessments. ( Domain E) 

■ Principal reports that the district emphasizes early identification of students needing 
academic support and addresses the needs of students two or more years behind grade 
level. ( Domain F) 

■ Principal and teachers report that the district provides useful professional development for 
teachers. ( Domain H) 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Principal reports that she/he meets with district administrators to review the school’s CST 
scores, benchmark test scores, AYP and API progress, and grades and/or grading policy. 

( Domain I) 

The study also found, for schools in the 70th-85th percentile of the SCI: 

■ Principal reports that the district leads decisions about the school’s 6th— 8th grade 
curriculum program adoption in mathematics. ( Finding unique to this domain) 
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Three domains, although positively correlated with CST outcomes, had the least predictive 
strength 

The three remaining domains also differentiated higher-performing middle grade schools 
in the sample, but they did so with less predictive strength than the seven domains already 
described. This was the case in almost every analysis performed, whether cross-sectional 
or longitudinal, for the “pooled” sample or the two SCI bands independently, or for all 
CST outcomes combined or by subject area. 

The three domains that were least associated with higher school-level CST outcomes 
were: 

■ Domain A: A positive, safe, engaging school environment. 

■ Domain C: School organization of time and instruction. 

■ Domain G: Attention to student transitions. 

The fact that these three domains had the least strength in predicting higher school-level 
outcomes on standards -based tests does not mean that they represent unimportant 
concerns, however. The following highlights aspects of these three domains that clearly 
differentiated higher -performing middle grade schools in the sample. 



48 



Initial Research Report — Gaining Ground in the Middle Grades 



Domain A-A positive, safe, engaging school environment 

Subdomains differentiating higher-performing schools under this domain pertain primarily 
to practices and policies focused on student attendance and behavior. For example, 
teachers’ reports of clear communication with students and parents about attendance and 
behavior, and of teacher collaboration to develop responses to problems, correlated with 
higher school-level achievement outcomes in both the cross-sectional and longitudinal 
analyses. The same was true of principal and teacher reports about the enforcement of 
school safety measures. 

Schools in which the principal reported larger proportions of students enrolled in 
extracurricular activities and electives also tended to have higher student achievement, 
across multiple outcomes and both the cross-sectional and longitudinal analyses. 



FINDINGS 

Domain A-A positive, safe, engaging school environment 

Other things being equal, stronger agreement about the following was associated with higher 
school performance 

Student behavior and attendance: 

■ Teachers report that: 

• The school clearly communicates rules and policies on behavior to students and 
parents. 

• Excellent student behavior and/or attendance gets classroom and/or schoolwide 
public recognition. 

• The importance of attendance and the consequences of frequent absenteeism, such 
as academic failure, are clearly communicated to students and parents. 

• Little of their time in class is spent managing student behavior. 

• Teachers in the school collaborate to develop strategies to address student behavior 
and/or attendance issues. 

A safe and positive school environment: 

■ Principal reports that the school enforces a comprehensive set of strategies to ensure a 
safe and positive school environment. 

■ Teachers report school enforcement of comprehensive safety strategies and that they feel 
safe in their school. 

Extracurricular activities and course electives: 

■ Principal reports that a high proportion of students participate in: 

• One or more extracurricular activities. 

• Course electives: Music, drama, art, dance. 

• Course electives: Exploratory wheel (providing for short explorations in several 
subjects during a one- or two-semester course) or mini-courses. 

The study also found, for schools in the 20th-35th percentile of the SCI: 

■ Principal reports that school staff/teachers accept shared responsibility for improving 
student attendance, such as by personally contacting students who cut classes and 
requiring detention for tardy students. 

■ Principal reports that school staff regularly analyze suspension data to ensure criteria are 
fairly applied to all students. 
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Domain C-School organization oftime and instruction 

Subdomains differentiating higher-performing schools under this domain pertain primarily 
to time as a key school resource. For example, schools in which the principal reported 
greater provision of common planning time per month in grades 7 and/or 8 tended to have 
higher achievement across a number of CST outcomes in those grades. 

In addition, some results regarding the positive role of subject-matter leadership emerged, 
at least for grade 8. The reported presence of subject-matter department chairs in English 
language aids and mathematics correlated positively with school-level outcomes on both 
the English Language Arts and Algebra I CSTs in grade 8, across both the cross-sectional 
and longitudinal analyses. 

This study cannot report clear and coherent positive findings pertaining to the internal 
organization of classroom instruction (e.g., self-contained vs. single-subject vs. semi self- 
contained classrooms) or master scheduling structures (e.g., 6-7 period days vs. block 
schedules), however. Although a number of subdomains pertaining to particular classroom 
organization and scheduling practices correlated positively with a few student outcomes, 
these results were confounding and frequently contradicted one another, and thus are not 
reported as findings below. Further descriptive observations regarding classroom 
organization as it relates to grade configuration are presented in the Additional Descriptive 
Findings and Observations of Note section of this report, however. 



FINDINGS 

Domain C-School organization oftime and instruction 

Other things being equal, stronger agreement about the following was associated with higher 

school performance 

Time for common planning: 

■ Principal reports a considerable amount of time allocated per month for common planning 
in grades 7 and/or 8 by English language arts and mathematics teachers. 

Instructional time beyond what the state requires: 

■ Principal reports that the school exceeds the state’s minimum annual requirements for 
instructional minutes/days. 

Provision of instructional time for different groups of English learner (EL) students: 

■ Principal reports that the school adjusts the amount of instructional time provided to EL 
students in English language arts, mathematics, and/or English language development 
based on students’ level of fluency, with “reasonably fluent” EL students spending less 
instructional time in these subjects than EL students with “less-than-reasonable” fluency. 

Subject-matter department chairs: 

■ Teachers report that the school has subject-matter department chairs in English language 
arts and mathematics. 
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Domain G— Attention to student transitions 

As noted elsewhere, several recent studies draw attention to the potential use of student 
data — e.g., academic, behavior, attendance — in the middle and even late elementary 
grades to provide “early warning” signals to educators about prospective academic or 
other problems in students’ subsequent years. The research team reviewed these studies 
and, in response, designed survey items and subdomains explicitly to study schools’ data 
practices as they pertain to student transitions into the middle grades. 

The findings reported for Domain G are consistent with and support these recent studies, 
and pertain directly to schools’ reported use of diverse student data in managing the 
transition into the middle grades. In particular, these findings highlight the importance of 
reviewing the academic, English proficiency, attendance, and behavior histories of 
incoming students and implementing academic support strategies when needed. 



FINDINGS 

Domain G- Attention to student transitions 

Other things being equal, stronger agreement about the following was associated with higher 
school performance 

Review of prior academic records, attendance, and behavior for students entering the 
middle grades: 

■ Principal and teachers report that school administrative staff and teachers: 

• Review prior CST scores. 

• Review prior English language arts and mathematics grades. 

• Review prior attendance records. 

• Review prior behavior reports. 

• Communicate with elementary teachers about any students entering the middle 
grades with low CST scores, failing English language arts or math grades, poor 
behavior reports, or poor attendance records. 

■ Teachers report that school administrative staff and teachers review prior CELDT 
(California English Language Development Test) scores and other English proficiency 
information for entering EL students. 

Classroom and intervention support for incoming students: 

■ Teachers report that school administrative staff and teachers implement classroom and 
intervention strategies to address academic weaknesses among incoming students. 
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Additional descriptive findings and observations of note 



Practices and policies pertaining to English learner (EL) students 

This study did not specifically examine the CST outcomes of middle grade English learner 
(EL) students enrolled in the participating schools. Rather, the research team only 
considered school-level CST outcomes for all students taking each of these tests in a 
school, regardless of students’ English proficiency. That said, some subdomains pertaining 
specifically to EL students did predict higher overall student achievement in the cross- 
sectional and longitudinal analyses. 

Several of these subdomains pertain to the messages districts send and the actions they 
take to demonstrate to local educators that meeting the needs of EL students is a priority 
for the middle grades. Specifically, other things being equal, overall achievement on 
various CST outcomes was higher in schools where: 

■ The principal reported more strongly that the district emphasizes the importance of 
English language development and subject-matter learning by EL students, and 
addresses the needs of EL students who are new immigrants; 

■ Teachers reported more strongly that the district ensures that policies and resources are 
in place to serve EL students; and 

■ The superintendent reported expecting middle grade schools to consider a range of 
factors — such as English and native language proficiency levels, prior academic 
performance, and time in California schools — when placing non-newcomer EL 
students into English language arts and mathematics classes. 

Similarly, overall achievement was higher in schools where teachers reported reviewing 
prior CELDT (California English Language Development Test) scores and other English 
proficiency information for entering EL students, and where principals reported that 
school staff and parents meet to develop intervention plans for individual EL students. 
Overall achievement on some outcomes was also higher to the extent that the principal 
reported having been trained in the evaluation of instruction for EL students and 
understanding the principles of EL second-language acquisition. 

A finding regarding instructional time for middle grade EL students raises questions for 
further study. Other things being equal, several 8th grade outcomes were higher in schools 
where the principal reported more consistently that the school adjusts the amount of 
instructional time provided to EL students in English language arts, mathematics, and/or 
English language development based on differences in students’ fluency levels. 
Specifically, these principals reported that “reasonably fluent” EL students do not spend as 
much time in these subjects as do EL students with “less than reasonable fluency.” This 
finding raises the possibility that there is an academic benefit when schools draw more 
nuanced distinctions among EL students with respect to English proficiency. It also 
suggests that middle grade schools that provide reasonably fluent EL students with more 
instruction in subjects other than English language arts and mathematics are more likely to 
be higher-achieving. 

Grade configuration and classroom organization 

Among middle grade educators in particular, questions often arise regarding the most 
effective grade configuration for serving students in the 6th, 7th, and 8th grades. Views 
differ about whether certain configurations offer educational advantages or if perceived 
advantages reflect other organizational or socioeconomic characteristics of schools. (For 
an introduction to the many different perspectives and concerns at stake in this debate see, 
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e.g., Cuban, 1992; Alspaugh, 1998b; Juvonen, Le, et al., 2004; Weiss and Kipnes, 2006; 
Byrnes & Ruby, 2007; Rubenstein, Schwartz, et al., 2009). Further, although K-8, 6-8, or 
7-8 configurations are sometimes adopted as educational strategies, decisions about grade 
configuration can also be driven by practical and political concerns, such as the 
availability of facilities, changing enrollment trends, and parental preferences. 

This study included schools with varying grade configurations. Among the 303 
participating schools: 

■ 73 (or 24%) were K-8 schools; 

■ 80 (or 26%) were 7-8 schools; and 

■ 150 (or almost 50%) were 6-8 schools, which includes a small number of schools with 
less common configurations, such as 5-8. 

After accounting for specific school policies and practices, no single grade configuration 
was consistently associated with higher performance on California's standards -based tests 
in English language arts and mathematics in this study. Both more effective and less 
effective policies and practices were found in schools with every grade configuration 
studied. 

However, the survey instruments used in this study do provide an opportunity to inquire 
descriptively into differences — if any — in how schools with these various grade 
configurations are organized internally. 

Popular wisdom often presumes that, on one end of the spectrum, K-8 schools tend to 
operate more like elementary schools while, on the other end of the spectrum, 7-8 schools 
operate more like high schools. Some of this study’s survey questions explored this. For 
example, principals were asked to report whether their schools have subject-matter 
departments for English language arts and mathematics. Only 31% of K-8 principals in 
the sample answered “yes,” compared with 94% of 6-8 principals and 98% of 7-8 
principals. 

Principals’ responses to another survey question — this one pertaining to classroom 
organization — are likely typical of the reality in many California middle grade schools. 

The survey asked principals to indicate — separately for classes in grades 6, 7, and 8 — 
whether classes were organized in one or more of the following ways: 

A. A different teacher for each subject area; 

B. One teacher for English language arts/history, one teacher for mathematics, and a 
separate teacher for science; 

C. Self-contained classrooms; 

D. One teacher for a block of English language arts/history and one teacher for a block 
of mathematics/science; or 

E. Other. 

In grade 6, the participating K-8 and 6-8 schools differ markedly in the extent to which 
principals reported exclusive use of self-contained classrooms (option C). In all, 48% of 
K-8 principals reported that 6th grade classrooms were exclusively self-contained, 
compared with only 8% of 6-8 principals. For both grade configurations, schools not 
exclusively using self-contained classrooms in grade 6 used a diverse range of reported 
alternatives. 

In grade 8, however, classes organized with a different teacher for each subject area 
(option A) appear to be the dominant strategy. As Figure 6 on the next page shows, K-8 
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schools employed a wide range of reported strategies for organizing 8th grade classrooms, 
while 6-8 and 7-8 schools tended to organize 8th grade classrooms in accordance with 
option A either exclusively or in combination with another organizational practice. 

These results hint at the extent to which such practices in many 6-8 and 7-8 schools in 
California may be more alike than different. Meanwhile, the reported practices in K-8 
schools in the sample are far from uniform; further examination of the survey items related 
to school organization could provide a clearer picture of how their practices vary. This is 
outside the scope of this Initial Research Report, however. 



Figure 6: How K-8, 6-8, and 7-8 schools in the sample organize classrooms in grade 8 



K-8 Schools 

(N - 72) 



6-8 Schools 

(N- 147) 



7-8 Schools 

(N - 80) 




KEY: 

□ A Only = Different Teacher for each 
subject area 

■ B Only = One teacher for ELA/ 
history, one teacher for math, and 
a separate teacher for science 



I C Only = Self contained 
classrooms 

1 0 Only = One teacher for a block of 
ELA/history and one teacher for a 
block of math/ science 



□ E Only = Other Organization 
Strategy 

■ Mixed = Multiple Organization 
Strategies Selected 



Source: EdSource Principal Survey data EdSource 2/10 

Total number of responding principals does not add up to 303 due to four missing responses. 

82% of K-8 principals who reported a mix of classroom organizational practices in grade 8 included option A; 
95% of 6-8 principals who reported a mix of classroom organizational practices in grade 8 included option A; 
and 100% of 7-8 principals who reported a mix of classroom organizational practices in grade 8 included 
option A. 
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V. Summary of findings and conclusion 

Taken as a whole, the findings presented in the previous section tell a coherent and 
compelling story. They highlight practices and policies that researchers, policymakers, and 
educators might consider when establishing plans for further improvement in the middle 
grades. 

Other things being equal, school-level achievement in the 303 -school sample tended to be 
higher to the extent that superintendents, principals, and middle grade English language 
arts and mathematics teachers strongly agreed on a clear, consistent, and intense focus on 
improving student academic outcomes. Further, the combined findings suggest a range of 
interrelated practices through which this focus happens. For example: 

■ There is a strong organizational press toward improving student outcomes, with 
superintendents, principals, and teachers emphasizing and being evaluated based in 
part on such improvements. 

■ Middle grade educators closely align curricula, instruction, and assessments with the 
state’s K-12 academic content standards in core subjects. From the district to the 
principal and the teachers, student assessment and other data are used extensively to 
evaluate and improve teacher practice and student outcomes. 

■ Middle grade educators have a future orientation toward preparing students to succeed 
in a rigorous high school curriculum. In other words, educators’ focus on student 
outcomes is not geared only toward improving student achievement as measured by 
standards-based exams. 

■ Perhaps in recognition that they have only a few years to work with their students, 
middle grade educators implement comprehensive and targeted academic programs to 
identify quickly and intervene proactively with students who are two or more years 
behind grade level, are English learners, or are at risk of failure in the current school 
year. 

■ A strong and cohesive professional culture marshals and focuses time and other 
resources intently on improving middle grade student outcomes. In particular: 

* The superintendent establishes an academic vision and follows through with 
accountability, while at the helm of an effectively functioning district that provides 
middle grade schools with comprehensive support aimed at improved student 
outcomes. 

* Principals provide strong leadership to drive and orchestrate a schoolwide focus on 
improved student outcomes. 

* Teachers work collectively as a team and individually in their classrooms on a 
shared mission to improve student outcomes and prepare students for success in 
high school. 

Notably, these practices and policies generally differentiated higher- from lower- 
performing middle grade schools in the sample regardless of whether schools were located 
in the 20th-35th or 70th-85th percentiles of the School Characteristics Index (SCI). 
Subdomains that differentiated higher-performing schools in only one of the two SCI 
bands across both the cross-sectional and longitudinal analyses were the exception rather 
than the rule. (Subdomains fitting this description are identified as such in the Findings 
section of this report.) 

Equally importantly, the district and school practices detailed in the Findings section as 
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correlating with higher school achievement in the middle grades can be adopted and 
implemented by any district, principal, and school community of teachers. They are not 
dependent upon grade configuration or internal classroom organization. 

Certainly, the California Standards Tests in English language arts and mathematics are not 
the only way for middle grade schools in the state to measure how well their students are 
mastering the academic content of the state’s standards. They also do not measure many 
other important things that middle grade students should be learning at school — such as 
art, social studies, science, and music, as well as citizenship and tolerance of differences. 

But scores on these tests do provide middle grade students and teachers, their school 
districts, and the state with a consistent way to measure the progress students are making 
toward mastering the important mathematics, reading, and writing skills that will enable 
them to succeed academically in high school and beyond. 

Across California — and no doubt nationwide — schools serving similar student populations 
can vary widely in how well their students perform on such tests. This study shows that, 
although the socioeconomic backgrounds of students are one strong predictor of school- 
level academic achievement, the practices and policies enacted by middle grade educators 
also have a significant relationship with these outcomes. Educators’ focus on their middle 
grade mission, and the resources they have available to pursue their goals, can make a 
difference. The interrelated practices identified in this study may help middle grade 
schools and districts — in California and nationally — continue their efforts to improve 
students’ academic outcomes at a critical time in their academic careers. 



VI. Further studies 



The scope and complexity of this study made it impossible for the research team to inquire 
deeply into every important topic about which it collected survey data. This leaves 
opportunities for further inquiry on timely topics — such as middle grade policies and 
practices in mathematics, including Algebra I — that could further enrich the discussion of 
these critical years of education. 

In the spring and summer of 2010, the research team will conduct a deeper, more detailed 
analysis of middle grade mathematics and Algebra I outcomes and their relationships with 
school placement policies, student participation rates in Algebra I, school choice of 
curriculum, teacher credentials and other subject-matter qualifications, and other relevant 
practices and policies. 

If funded, another analysis could explore the possibility of tracking the 8th graders in this 
study into high school to examine their 9th-grade mathematics placements and CST scores 
to see what these reveal about the impact of middle grade mathematics in high school. 

EdSource also wants to follow up with fieldwork and case studies to more deeply 
understand how some districts and schools — especially those previously but no longer in 
Program Improvement under the federal No Child Left Behind law — were able to develop 
alignment around and an intense focus on student outcomes. 
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VII. Implications for practice and policy 

The research team believes this report makes an important contribution to the field by 
identifying policies and practices that correlate with higher achievement in middle grade 
schools. The team invites educators and policymakers to use this collection of rich 
findings to evaluate their own practices and guide conversations about reform at the state, 
district, and school levels. Some general recommendations based on the report’s findings 
are: 

■ Local educators can use these findings to learn more about what is working in 
some higher-performing schools and as the basis for staff discussions about ways 
to improve student outcomes in their own schools. 

■ State policymakers should examine the extent to which current state policies and 
budget cuts either strengthen or inhibit local schools’ and districts’ ability to 
carry out the practices this study found to be significant. 

While further research and discussion are warranted, the research team wants to highlight 
the following implications for consideration in the context of policy development, 
evaluation of current practices, and improving reform strategies. 

Implications for districts and schools 

There may be good reasons for deciding to have a K-8, 6-8, or 7-8 grade 
configuration, but improving student outcomes is not necessarily one of them. After 
accounting for specific school policies and practices, no single grade configuration was 
consistently associated with higher performance on the state’s standards-based tests in 
English language arts and mathematics. 

Superintendents and boards overseeing grades 6-8 students should discuss the 
priority they give to academic improvements in the middle grades. This study clearly 
documents the positive influence of district superintendents and boards on middle grade 
student academic outcomes when they communicate the importance of outcomes; include 
those outcomes in their annual evaluations; provide leadership and policies around 
curriculum and other instructional practices; and align resources to meet academic goals. 

The results of this study should encourage principals to engage their staff members 
and their teachers in conversations about their mission for the middle grades in their 
schools. Educators in the middle grades have long believed that responsiveness to early 
adolescent developmental issues and strong adult-student relationships are a central part of 
the middle grades imperative. They are, but so is academic learning. Educators from 
higher-performing schools in this study provided a safe and positive environment, 
extracurricular activities, and frequent efforts to reach out to students and parents. 
However, the central focus of their collective time and energy was on the kinds of 
strategies — extensive review and use of data, proactive student interventions, and 
standards-based instruction — that are associated with improved student learning and 
outcomes in English language arts and mathematics. 

Prioritize strategies for helping students make gains on standards-based exams in the 
context of the middle grades’ unique position in the K-12 hierarchy to prepare all 
students to succeed in high school. Students who do well on standards-based exams in 
English language arts and mathematics in the middle grades are more likely to pass 
California’s high school exit exam and graduate. Students who do well on 7th and 8th 
grade mathematics CSTs are more likely to start high school in a college preparatory 
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curriculum. If the school views improved student outcomes on standards-based exams as a 
priority because students benefit, then the school mission will reflect that belief. 

When hiring middle grade principals, districts should consider looking for the kind 
of skills and competencies found in principals of the higher-performing schools in 
this study. In addition, the district can ensure that a principal has the authority to replace 
the school leadership team, if necessary, with one that can better support him/her in 
driving change. 

When hiring middle grade teachers, districts and principals should consider looking 
for the kinds of interests, skills, and competencies that principals in higher- 
performing schools report about their teachers. And if those competencies are present, 
this study suggests that many teachers asked to work at the middle grade level will grow to 
love it. Of the 3,752 teachers surveyed for this study, only 1,609 entered teaching with the 
intention of working with middle grade students — but 3 ,247 now report the middle grades 
as their current first choice of teaching assignment. 

Examine the extent to which their middle grade curricula, instructional practices, 
and assessments are tightly aligned with state academic standards. Questions 
include: Does teacher professional development help teachers map key state standards by 
grade and subject to instruction? Does the district provide standards-based benchmark 
tests for middle grades and return results quickly? Do teachers work collectively and know 
how to respond to test results? Do teachers use diagnostic tests to determine why a student 
is struggling? Is there support to help teachers address students’ instructional needs? 

Consider making improvements in middle grade student outcomes a part of 
professional educator performance evaluations. Consistent with much of the national 
conversation around educator effectiveness, this study found that evaluating 
superintendents, principals, and teachers in part on improvements in student outcomes was 
associated with higher student outcomes on standards-based exams. This practice should 
be part of a comprehensive strategy that also includes availability of student data, 
meaningful professional development for teachers and principals, and schools’ possessing 
a complete portfolio of student intervention strategies. This study did not find that salary 
adjustments for superintendents or principals based upon student performance, which is 
not a common practice in California, was associated with higher student outcomes. The 
survey did not ask a question about teacher salary adjustment based on student outcomes. 

Implications for state policy 

California policymakers should make it a priority to do what they can to sustain the 
state’s investment in public education, including a focus on the middle grades. The 

middle grades are critical to student success in high school. And despite California having 
fewer resources than many other states, this study shows that many middle grade schools 
here are succeeding at improving student outcomes. This study has identified practices 
associated with these higher-performing schools — such as frequent and adequate time for 
common planning, a comprehensive array of student intervention strategies as well as 
extracurricular classes, access to timely student assessment data, and the computer 
software and training to effectively use it to improve instruction and learning. Each of 
these requires resources that are difficult to find when budgets are being cut. State 
policymakers should do all they can during these challenging fiscal times to continue to 
support middle grade schools in their efforts to improve student outcomes. 

The practices in this study reflect a positive intersection between state policy and the 
schools’ ability to implement it effectively. It has taken time, but California’s aligned 
standards-based reforms are taking hold and are reflected in the higher-performing schools 
no matter what the socioeconomic background of their students. California should stay the 
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course, completing the reforms where there are gaps, refining the system where 
improvements would have the most impact, and doing all it can to provide local educators 
with critical support. 

California should be thoughtful when refining its K-12 academic content standards. 

This study’s findings make it clear that higher-performing middle grade schools use the 
state’s academic standards, and the adopted curriculum programs, to support their efforts 
to improve student outcomes and prepare students for high school. This momentum should 
be taken into consideration as state officials consider changes to the state’s academic 
standards. That said, higher-performing middle grade schools also report that they identify 
and focus on select “key standards” — those most critical to master — at each grade level 
and in each subject. A state policy that keeps standards rigorous but moves toward fewer 
standards of greater depth might especially help lower-performing schools better focus 
their efforts. 

Completing California’s student data system, combined with support to help districts 
effectively access and use the data, must remain a high priority. Effective use of data 
can make a difference in student outcomes, but the ability to access and use data varies 
widely across the state. Although California and its school districts have been developing 
data capacity for years, criticism lingers that the state has not invested enough to make this 
a reality for its nearly 1,000 school districts. 

Based on this study, student academic interventions at the middle grade level are an 
essential tool for educators to get on track the students who are behind grade level, 
students who are growing disengaged, and English learners. California’s budget cuts 
to K-12 education this past year and looking forward are likely to have a seriously 
negative effect on schools’ ability to provide an array of effective intervention strategies 
for all the types of students, including English learners, who could benefit. Districts and 
schools trying to improve student outcomes will need state, federal, or philanthropic 
support to maintain effective required and voluntary interventions. 

Implications for federal policy 

The findings reinforce several Race to the Top (RTT) central principles for 
improving student outcomes, in particular rigorous standards and quality 
assessments, and the use of student data to improve teaching and learning. 

California might highlight the following in its future discussions with the U.S. 
Department of Education about its RTT grant program: 

■ RTT requires states to demonstrate they are making progress toward rigorous college- 
and career-ready standards and quality assessments. The state’s higher-performing 
middle grade schools exemplify these principles: they report that the state’s academic 
standards are the foundation for high expectations for all students, rigorous 
instructional programs, and student assessments. 

■ Similarly, principals and teachers in these districts and schools use data and data 
systems as envisioned by RTT. They report that districts provide data in a timely 
manner using systems that allow easy access and management. These schools use data 
extensively to guide instruction, identify student needs, and improve teacher practices. 

■ In two other RTT areas — turning around low-performing schools and distributing high- 
quality teachers and leaders in equitable ways — the findings point to policies and 
practices that support these priorities. For example, the state’s higher-performing 
middle grade schools report that their districts provide useful professional 
development, ensure teachers are assigned so that students with the greatest need are 
served well, and give principals the opportunity to reconstitute leadership teams. 
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For federal policymakers, the findings can help inform the reauthorization of the 
Elementary and Secondary Education Act in several areas: 

■ The term “college ready” does not apply only to what happens in high schools. This 
study makes it clear that higher-performing middle grade schools think about and plan 
for a rigorous high school curriculum as a foundation for college- and career-readiness. 

■ The early identification of struggling students and appropriate interventions — 
particularly in the middle grades — are priorities for California. These findings offer 
examples for how federal policy and spending could reflect these priorities in any new 
education legislation. 

Implications for research 

As both benchmarking and diagnostic assessments become more common, 
researchers, K-12 educators, and state policy groups should carefully review the 
quality and validity of these tools. Further development and improvement in these types 
of assessments could strengthen their usefulness for teachers in their daily work and for 
adjusting their practice. 

Current efforts by foundations, researchers, and federal policymakers to develop a 
new definition of what makes an effective teacher could contribute significantly to 
the field. Teacher effectiveness, based upon a list of attributes and competencies cited by 
principals, was strongly associated with higher student achievement. This study did not 
find that a specific credential was associated with higher school performance, but it did 
find that other attributes — such as teacher subject-matter knowledge and skills in using 
assessment data to identify student needs — were important. 

Principal training and certification programs should be reviewed to ensure that they 
prepare leaders who can meet the needs of local schools. This study is consistent with 
others in recent years that chronicle the changing role of principals in higher-performing 
schools to that of manager of school improvement and change, and driver of an 
orchestrated and coordinated effort by the whole staff to boost outcomes for all students. 

Concluding thoughts 

The effective middle grade practices reflected in this study are actionable and replicable. 
They can serve as a kind of research-based checklist against which educators wanting to 
improve their student outcomes can compare what they are doing to ensure academic 
success to the effective practices reported by higher-performing middle grade schools in 
California. 

The study's findings also provide valuable information to California state policy leaders 
and those in other states as they consider refining state academic standards and other 
policies, targeting investments, and improving how data are collected and provided. 

The research team is confident that though the findings are from California schools, they 
have national applicability. The team invites educators and policymakers nationally and in 
Washington, DC, to review what this EdSource study found and to draw ideas and 
inspiration from this work to help make all middle grade schools high performers. 

Finally, the research team also invites discussion and suggestions for additional 
implications of the findings and for meaningful follow-up studies that could make a 
contribution to the field and to the knowledge base of reformers and policymakers. 
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